[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:01:33 dist-test-slave-8tdl krb5kdc[14148](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:01:33 dist-test-slave-8tdl krb5kdc[14148](info): set up 2 sockets
Aug 08 07:01:33 dist-test-slave-8tdl krb5kdc[14148](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 07:01:34.220804 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:01:34 dist-test-slave-8tdl krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:01:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:01:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:01:34.366914 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:32845
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:42031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:32845
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:34.457296 14161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:34.457584 14161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:34.457630 14161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:34.458869 14161 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:01:34.458945 14161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:34.458967 14161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:34.458988 14161 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:01:34.459020 14161 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:01:34.460579 14161 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:42031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:32845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:32845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:34.460901 14161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:34.461153 14161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:34.466895 14166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:34.466912 14161 server_base.cc:1034] running on GCE node
W20260808 07:01:34.466883 14169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:34.466877 14171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:34.467581 14161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:34.468019 14161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:34.469300 14161 hybrid_clock.cc:648] HybridClock initialized: now 1786172494469286 us; error 65 us; skew 500 ppm
Aug 08 07:01:34 dist-test-slave-8tdl krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172494, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:34.472414 14161 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:01:34.473003 14161 webserver.cc:460] Webserver started at http://127.13.207.62:36191/ using document root <none> and password file <none>
I20260808 07:01:34.473274 14161 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:34.473348 14161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:34.473512 14161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:34.474996 14161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "3d17d00a7221487a9c365d40c02c35f1"
format_stamp: "Formatted at 2026-08-08 07:01:34 on dist-test-slave-8tdl"
server_key: "a54d758537db1e7ec0ad9c1041a93015"
server_key_iv: "280d1478ab17c0ef98c7f6f6d6ce98e4"
server_key_version: "encryptionkey@0"
I20260808 07:01:34.475416 14161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "3d17d00a7221487a9c365d40c02c35f1"
format_stamp: "Formatted at 2026-08-08 07:01:34 on dist-test-slave-8tdl"
server_key: "a54d758537db1e7ec0ad9c1041a93015"
server_key_iv: "280d1478ab17c0ef98c7f6f6d6ce98e4"
server_key_version: "encryptionkey@0"
I20260808 07:01:34.477434 14161 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260808 07:01:34.478479 14178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:34.478806 14161 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:34.479012 14161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "3d17d00a7221487a9c365d40c02c35f1"
format_stamp: "Formatted at 2026-08-08 07:01:34 on dist-test-slave-8tdl"
server_key: "a54d758537db1e7ec0ad9c1041a93015"
server_key_iv: "280d1478ab17c0ef98c7f6f6d6ce98e4"
server_key_version: "encryptionkey@0"
I20260808 07:01:34.479118 14161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:34.502576 14161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:34.505506 14161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:34.505769 14161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:34.511142 14161 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:32845
I20260808 07:01:34.511238 14232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:32845 every 8 connection(s)
I20260808 07:01:34.511667 14161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:01:34.512853 14233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:34.514074 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 14161
I20260808 07:01:34.514168 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:01:34.514922 14140 external_mini_cluster.cc:1428] Setting key 8f675faf1df13454ea87b63a6b831a3f
I20260808 07:01:34.516010 14233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d17d00a7221487a9c365d40c02c35f1: Bootstrap starting.
I20260808 07:01:34.516723 14233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d17d00a7221487a9c365d40c02c35f1: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:34.517042 14233 log.cc:824] T 00000000000000000000000000000000 P 3d17d00a7221487a9c365d40c02c35f1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:34.517863 14233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d17d00a7221487a9c365d40c02c35f1: No bootstrap required, opened a new log
I20260808 07:01:34.519801 14233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d17d00a7221487a9c365d40c02c35f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d17d00a7221487a9c365d40c02c35f1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 32845 } }
I20260808 07:01:34.520002 14233 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d17d00a7221487a9c365d40c02c35f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:34.520056 14233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d17d00a7221487a9c365d40c02c35f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d17d00a7221487a9c365d40c02c35f1, State: Initialized, Role: FOLLOWER
I20260808 07:01:34.520236 14233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d17d00a7221487a9c365d40c02c35f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d17d00a7221487a9c365d40c02c35f1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 32845 } }
I20260808 07:01:34.520340 14233 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d17d00a7221487a9c365d40c02c35f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:01:34.520375 14233 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d17d00a7221487a9c365d40c02c35f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:01:34.520421 14233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d17d00a7221487a9c365d40c02c35f1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:34.521804 14233 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d17d00a7221487a9c365d40c02c35f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d17d00a7221487a9c365d40c02c35f1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 32845 } }
I20260808 07:01:34.521958 14233 leader_election.cc:302] T 00000000000000000000000000000000 P 3d17d00a7221487a9c365d40c02c35f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d17d00a7221487a9c365d40c02c35f1; no voters:
I20260808 07:01:34.522176 14233 leader_election.cc:288] T 00000000000000000000000000000000 P 3d17d00a7221487a9c365d40c02c35f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:01:34.522336 14238 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d17d00a7221487a9c365d40c02c35f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:34.522552 14233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d17d00a7221487a9c365d40c02c35f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:01:34.522560 14238 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d17d00a7221487a9c365d40c02c35f1 [term 1 LEADER]: Becoming Leader. State: Replica: 3d17d00a7221487a9c365d40c02c35f1, State: Running, Role: LEADER
I20260808 07:01:34.522716 14238 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d17d00a7221487a9c365d40c02c35f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d17d00a7221487a9c365d40c02c35f1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 32845 } }
I20260808 07:01:34.523300 14240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d17d00a7221487a9c365d40c02c35f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d17d00a7221487a9c365d40c02c35f1. Latest consensus state: current_term: 1 leader_uuid: "3d17d00a7221487a9c365d40c02c35f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d17d00a7221487a9c365d40c02c35f1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 32845 } } }
I20260808 07:01:34.523396 14240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d17d00a7221487a9c365d40c02c35f1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:34.523762 14239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d17d00a7221487a9c365d40c02c35f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d17d00a7221487a9c365d40c02c35f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d17d00a7221487a9c365d40c02c35f1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 32845 } } }
I20260808 07:01:34.523852 14239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d17d00a7221487a9c365d40c02c35f1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:34.524197 14244 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:01:34.524711 14244 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:01:34.527163 14244 catalog_manager.cc:1347] Generated new cluster ID: 143e0760b14842ea8a56751760a2f469
I20260808 07:01:34.527258 14244 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:01:34 dist-test-slave-8tdl krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:34.537714 14236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:34.516038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38076 (local address 127.13.207.62:32845)
0808 07:01:34.516270 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:34.516279 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:34.516529 (+ 250us) server_negotiation.cc:407] Connection header received
0808 07:01:34.519796 (+ 3267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:34.519806 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:34.519870 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:34.520149 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:34.522752 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:34.524652 (+ 1900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:34.526853 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:34.527602 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:34.527729 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:34.532916 (+ 5187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:34.532921 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:34.532922 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:34.532937 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:34.535143 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:34.535809 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:34.535812 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:34.535813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:34.535952 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:34.536312 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:34.536313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:34.536313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:34.536655 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:34.536796 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:34.537240 (+ 444us) server_negotiation.cc:299] Negotiation successful
0808 07:01:34.537452 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":107,"threads_started":1}
I20260808 07:01:34.559777 14244 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:01:34.560549 14244 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:01:34.576057 14244 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d17d00a7221487a9c365d40c02c35f1: Generated new TSK 0
I20260808 07:01:34.576357 14244 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
I20260808 07:01:34.623312 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:32845
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:42031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:34.717162 14259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:34.717463 14259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:34.717497 14259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:34.719739 14259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:34.719808 14259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:34.719851 14259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:01:34.721869 14259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:42031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:32845
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:34.722174 14259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:34.722465 14259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:34.728873 14264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:34.728869 14265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:34.728907 14259 server_base.cc:1034] running on GCE node
W20260808 07:01:34.728865 14267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:34.729315 14259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:34.729768 14259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:34.730960 14259 hybrid_clock.cc:648] HybridClock initialized: now 1786172494730935 us; error 47 us; skew 500 ppm
Aug 08 07:01:34 dist-test-slave-8tdl krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172494, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:34.733649 14259 init.cc:377] Logged in from keytab as kudu/127.13.207.1@KRBTEST.COM (short username kudu)
I20260808 07:01:34.734138 14259 webserver.cc:460] Webserver started at http://127.13.207.1:33187/ using document root <none> and password file <none>
I20260808 07:01:34.734377 14259 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:34.734447 14259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:34.734613 14259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:34.735762 14259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "69076fe694b14434b0fcf20d09128fc4"
format_stamp: "Formatted at 2026-08-08 07:01:34 on dist-test-slave-8tdl"
server_key: "e8ef5f1eaf93b76fcd8b4821bd7743ad"
server_key_iv: "ff4846dd28aa70164db8c8901f96689e"
server_key_version: "encryptionkey@0"
I20260808 07:01:34.736390 14259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "69076fe694b14434b0fcf20d09128fc4"
format_stamp: "Formatted at 2026-08-08 07:01:34 on dist-test-slave-8tdl"
server_key: "e8ef5f1eaf93b76fcd8b4821bd7743ad"
server_key_iv: "ff4846dd28aa70164db8c8901f96689e"
server_key_version: "encryptionkey@0"
I20260808 07:01:34.737931 14259 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:01:34.738965 14274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:34.739218 14259 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:34.739338 14259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "69076fe694b14434b0fcf20d09128fc4"
format_stamp: "Formatted at 2026-08-08 07:01:34 on dist-test-slave-8tdl"
server_key: "e8ef5f1eaf93b76fcd8b4821bd7743ad"
server_key_iv: "ff4846dd28aa70164db8c8901f96689e"
server_key_version: "encryptionkey@0"
I20260808 07:01:34.739423 14259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:34.748334 14259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:34.751798 14259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:34.751987 14259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:34.752267 14259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:34.752653 14259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:34.752715 14259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:34.752763 14259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:34.752789 14259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:34.760711 14259 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:39893
I20260808 07:01:34.760756 14391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:39893 every 8 connection(s)
I20260808 07:01:34.761201 14259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:01:34 dist-test-slave-8tdl krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172494, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:34.770524 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 14259
I20260808 07:01:34.770613 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:01:34.770815 14140 external_mini_cluster.cc:1428] Setting key c2c5753485b99d45e7a1620b975d6987
I20260808 07:01:34.773535 14236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:34.762206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:40877 (local address 127.13.207.62:32845)
0808 07:01:34.762305 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:34.762307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:34.762499 (+ 192us) server_negotiation.cc:407] Connection header received
0808 07:01:34.763155 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:34.763156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:34.763173 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:34.763246 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:34.764606 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:34.765164 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:34.766421 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:34.766655 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:34.766749 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:34.769826 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:34.769828 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:34.769829 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:34.769839 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:34.771850 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:34.772407 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:34.772408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:34.772408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:34.772481 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:34.772766 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:34.772767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:34.772767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:34.772903 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:34.772985 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:34.773317 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 07:01:34.773446 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 07:01:34.773769 14395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:34.762073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:32845 (local address 127.13.207.1:40877)
0808 07:01:34.762367 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:34.762414 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:34.763027 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:34.763389 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:34.763394 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:34.764199 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:34.764449 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:34.764455 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:34.765324 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:34.765325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:34.766281 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:34.766285 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:34.766837 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:34.766838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:34.766981 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:34.767768 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:34.767779 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:34.769650 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:34.772003 (+ 2353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:34.772015 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:34.772017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:34.772307 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:34.772562 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:34.772563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:34.772568 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:34.772695 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:34.772975 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:34.772983 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:34.773135 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 07:01:34.773258 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 07:01:34.773475 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":210,"threads_started":1}
I20260808 07:01:34.774312 14393 heartbeater.cc:347] Connected to a master server at 127.13.207.62:32845
I20260808 07:01:34.774415 14393 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:34.774636 14393 heartbeater.cc:511] Master 127.13.207.62:32845 requested a full tablet report, sending...
I20260808 07:01:34.775279 14197 ts_manager.cc:194] Registered new tserver with Master: 69076fe694b14434b0fcf20d09128fc4 (127.13.207.1:39893)
I20260808 07:01:34.775936 14197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:40877
WARNING: no policy specified for kudu/127.13.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
I20260808 07:01:34.807140 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:32845
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:42031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:34.903611 14398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:34.903877 14398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:34.903909 14398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:34.905268 14398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:34.905320 14398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:34.905354 14398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:01:34.907146 14398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:42031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:32845
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:34.907397 14398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:34.907621 14398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:34.913357 14403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:34.913384 14404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:34.913357 14406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:34.913887 14398 server_base.cc:1034] running on GCE node
I20260808 07:01:34.914088 14398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:34.914328 14398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:34.915516 14398 hybrid_clock.cc:648] HybridClock initialized: now 1786172494915471 us; error 36 us; skew 500 ppm
Aug 08 07:01:34 dist-test-slave-8tdl krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172494, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:34.918287 14398 init.cc:377] Logged in from keytab as kudu/127.13.207.2@KRBTEST.COM (short username kudu)
I20260808 07:01:34.919024 14398 webserver.cc:460] Webserver started at http://127.13.207.2:38675/ using document root <none> and password file <none>
I20260808 07:01:34.919279 14398 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:34.919351 14398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:34.919525 14398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:34.920646 14398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "7686d23c812d455d98c3a4105d72aad4"
format_stamp: "Formatted at 2026-08-08 07:01:34 on dist-test-slave-8tdl"
server_key: "8605767709b741fa3cbda5aa7cbd7063"
server_key_iv: "90fa040857c3e5514a3192d4f4a9b666"
server_key_version: "encryptionkey@0"
I20260808 07:01:34.921036 14398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "7686d23c812d455d98c3a4105d72aad4"
format_stamp: "Formatted at 2026-08-08 07:01:34 on dist-test-slave-8tdl"
server_key: "8605767709b741fa3cbda5aa7cbd7063"
server_key_iv: "90fa040857c3e5514a3192d4f4a9b666"
server_key_version: "encryptionkey@0"
I20260808 07:01:34.922603 14398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:01:34.923818 14414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:34.924180 14398 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:34.924309 14398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "7686d23c812d455d98c3a4105d72aad4"
format_stamp: "Formatted at 2026-08-08 07:01:34 on dist-test-slave-8tdl"
server_key: "8605767709b741fa3cbda5aa7cbd7063"
server_key_iv: "90fa040857c3e5514a3192d4f4a9b666"
server_key_version: "encryptionkey@0"
I20260808 07:01:34.924412 14398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:34.945706 14398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:34.948866 14398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:34.949110 14398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:34.949414 14398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:34.949784 14398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:34.949849 14398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:34.949887 14398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:34.949904 14398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:34.957502 14398 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:33827
I20260808 07:01:34.957551 14531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:33827 every 8 connection(s)
I20260808 07:01:34.957998 14398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:01:34.964483 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 14398
I20260808 07:01:34.964609 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:01:34.964808 14140 external_mini_cluster.cc:1428] Setting key ac2f5c5d239d6bd016978f8056975a49
Aug 08 07:01:34 dist-test-slave-8tdl krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172494, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:34.970846 14236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:34.959139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:47171 (local address 127.13.207.62:32845)
0808 07:01:34.959231 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:34.959234 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:34.959533 (+ 299us) server_negotiation.cc:407] Connection header received
0808 07:01:34.960184 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:34.960186 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:34.960197 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:34.960259 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:34.961543 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:34.962152 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:34.963560 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:34.963775 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:34.963854 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:34.967580 (+ 3726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:34.967582 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:34.967582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:34.967592 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:34.969133 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:34.969639 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:34.969641 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:34.969642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:34.969707 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:34.970021 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:34.970022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:34.970023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:34.970169 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:34.970256 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:34.970581 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 07:01:34.970744 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 07:01:34.971115 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:34.959088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:32845 (local address 127.13.207.2:47171)
0808 07:01:34.959291 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:34.959337 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:34.960021 (+ 684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:34.960459 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:34.960463 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:34.961180 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:34.961399 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:34.961403 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:34.962281 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:34.962282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:34.963404 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:34.963408 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:34.963921 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:34.963922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:34.964320 (+ 398us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:34.965258 (+ 938us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:34.965276 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:34.967454 (+ 2178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:34.969245 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:34.969255 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:34.969258 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:34.969542 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:34.969806 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:34.969807 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:34.969808 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:34.969937 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:34.970256 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:34.970266 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:34.970428 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 07:01:34.970548 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 07:01:34.970779 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":119,"threads_started":1}
I20260808 07:01:34.971972 14532 heartbeater.cc:347] Connected to a master server at 127.13.207.62:32845
I20260808 07:01:34.972131 14532 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:34.972345 14532 heartbeater.cc:511] Master 127.13.207.62:32845 requested a full tablet report, sending...
I20260808 07:01:34.973129 14197 ts_manager.cc:194] Registered new tserver with Master: 7686d23c812d455d98c3a4105d72aad4 (127.13.207.2:33827)
I20260808 07:01:34.973608 14197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:47171
WARNING: no policy specified for kudu/127.13.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
I20260808 07:01:34.998358 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:32845
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:42031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:35.094544 14537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:35.095225 14537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:35.095268 14537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:35.096604 14537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:35.096664 14537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:35.096696 14537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:01:35.098333 14537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:42031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:32845
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:35.098568 14537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:35.098842 14537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:35.104998 14543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:35.105090 14542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:35.105226 14537 server_base.cc:1034] running on GCE node
W20260808 07:01:35.105032 14545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:35.105763 14537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:35.106042 14537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:35.107267 14537 hybrid_clock.cc:648] HybridClock initialized: now 1786172495107250 us; error 31 us; skew 500 ppm
Aug 08 07:01:35 dist-test-slave-8tdl krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172495, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:35.117887 14537 init.cc:377] Logged in from keytab as kudu/127.13.207.3@KRBTEST.COM (short username kudu)
I20260808 07:01:35.118502 14537 webserver.cc:460] Webserver started at http://127.13.207.3:38563/ using document root <none> and password file <none>
I20260808 07:01:35.118811 14537 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:35.118891 14537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:35.119060 14537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:35.120469 14537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "022e7b83168742f0af5c6d4cdcb81926"
format_stamp: "Formatted at 2026-08-08 07:01:35 on dist-test-slave-8tdl"
server_key: "ca025d3d5e237587f3d9fb23ad409565"
server_key_iv: "d46f7a84f3fef6115e30fa5019252472"
server_key_version: "encryptionkey@0"
I20260808 07:01:35.121042 14537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "022e7b83168742f0af5c6d4cdcb81926"
format_stamp: "Formatted at 2026-08-08 07:01:35 on dist-test-slave-8tdl"
server_key: "ca025d3d5e237587f3d9fb23ad409565"
server_key_iv: "d46f7a84f3fef6115e30fa5019252472"
server_key_version: "encryptionkey@0"
I20260808 07:01:35.122956 14537 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 07:01:35.124161 14553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:35.124451 14537 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:35.124568 14537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "022e7b83168742f0af5c6d4cdcb81926"
format_stamp: "Formatted at 2026-08-08 07:01:35 on dist-test-slave-8tdl"
server_key: "ca025d3d5e237587f3d9fb23ad409565"
server_key_iv: "d46f7a84f3fef6115e30fa5019252472"
server_key_version: "encryptionkey@0"
I20260808 07:01:35.124679 14537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:35.164830 14537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:35.167850 14537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:35.168095 14537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:35.168411 14537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:35.168802 14537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:35.168867 14537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:35.168929 14537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:35.168958 14537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:35.176565 14537 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:44665
I20260808 07:01:35.176647 14667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:44665 every 8 connection(s)
I20260808 07:01:35.177085 14537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:01:35 dist-test-slave-8tdl krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172495, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:35.187100 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 14537
I20260808 07:01:35.187179 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:01:35.187363 14140 external_mini_cluster.cc:1428] Setting key e028771774095fadd9f3d109876abf4f
I20260808 07:01:35.189518 14236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:35.178096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:52555 (local address 127.13.207.62:32845)
0808 07:01:35.178208 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:35.178211 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:35.178434 (+ 223us) server_negotiation.cc:407] Connection header received
0808 07:01:35.179182 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:35.179183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:35.179198 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:35.179247 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:35.180640 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:35.181202 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:35.182380 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:35.182634 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:35.182733 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:35.185491 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:35.185493 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:35.185493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:35.185502 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:35.187452 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:35.188140 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:35.188142 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:35.188142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:35.188241 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:35.188619 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:35.188620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:35.188621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:35.188789 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:35.188970 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:35.189325 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 07:01:35.189441 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 07:01:35.189659 14670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:35.178075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:32845 (local address 127.13.207.3:52555)
0808 07:01:35.178276 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:35.178322 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:35.179071 (+ 749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:35.179318 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:35.179322 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:35.180259 (+ 937us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:35.180486 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:35.180492 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:35.181316 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:35.181317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:35.182277 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:35.182281 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:35.182759 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:35.182760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:35.182909 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:35.183787 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:35.183796 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:35.185355 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:35.187641 (+ 2286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:35.187652 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:35.187655 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:35.188024 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:35.188355 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:35.188356 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:35.188356 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:35.188523 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:35.188916 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:35.188925 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:35.189147 (+ 222us) client_negotiation.cc:769] Sending connection context
0808 07:01:35.189260 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 07:01:35.189439 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":106,"threads_started":1}
I20260808 07:01:35.190241 14668 heartbeater.cc:347] Connected to a master server at 127.13.207.62:32845
I20260808 07:01:35.190378 14668 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:35.190534 14668 heartbeater.cc:511] Master 127.13.207.62:32845 requested a full tablet report, sending...
I20260808 07:01:35.190958 14197 ts_manager.cc:194] Registered new tserver with Master: 022e7b83168742f0af5c6d4cdcb81926 (127.13.207.3:44665)
I20260808 07:01:35.191498 14197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:52555
I20260808 07:01:35.200647 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 07:01:35 dist-test-slave-8tdl krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172495, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:01:35 dist-test-slave-8tdl krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172495, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.207.1@KRBTEST.COM
I20260808 07:01:35.229449 14675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:35.219996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51710 (local address 127.13.207.1:39893)
0808 07:01:35.220215 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:35.220223 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:35.220309 (+ 86us) server_negotiation.cc:407] Connection header received
0808 07:01:35.220422 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:35.220425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:35.220441 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:35.220528 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:35.221246 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:35.222004 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:35.222899 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:35.223253 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:35.223357 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:35.225832 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:35.225844 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:35.225845 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:35.225853 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:35.228046 (+ 2193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:35.228460 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:35.228463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:35.228463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:35.228531 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:35.228739 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:35.228739 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:35.228740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:35.228956 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:35.229131 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:35.229222 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 07:01:35.229315 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":97,"threads_started":1}
W20260808 07:01:35.229878 14303 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51710
Aug 08 07:01:35 dist-test-slave-8tdl krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:01:35 dist-test-slave-8tdl krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.1@KRBTEST.COM
I20260808 07:01:35.263347 14675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:35.252668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51720 (local address 127.13.207.1:39893)
0808 07:01:35.252748 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:35.252751 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:35.252935 (+ 184us) server_negotiation.cc:407] Connection header received
0808 07:01:35.252981 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:35.252982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:35.252998 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:35.253052 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:35.253929 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:35.254503 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:35.255725 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:35.256189 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:35.256300 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:35.259658 (+ 3358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:35.259660 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:35.259660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:35.259671 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:35.261756 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:35.262327 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:35.262329 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:35.262329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:35.262404 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:35.262722 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:35.262724 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:35.262724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:35.262880 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:35.263009 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:35.263152 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 07:01:35.263260 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 07:01:35.263931 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 14259
I20260808 07:01:35.271345 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 14398
I20260808 07:01:35.277597 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 14537
I20260808 07:01:35.291273 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 14161
2026-08-08T07:01:35Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2245 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:01:35 dist-test-slave-8tdl krb5kdc[14682](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:01:35 dist-test-slave-8tdl krb5kdc[14682](info): set up 2 sockets
Aug 08 07:01:35 dist-test-slave-8tdl krb5kdc[14682](info): commencing operation
krb5kdc: starting...
W20260808 07:01:36.445994 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:01:36 dist-test-slave-8tdl krb5kdc[14682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:01:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:01:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:01:36.587316 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:41117
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:37441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:41117
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:36.677006 14695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:36.677394 14695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:36.677443 14695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:36.678815 14695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:01:36.678894 14695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:36.678917 14695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:36.678949 14695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:01:36.678980 14695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:01:36.680687 14695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:37441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:41117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:41117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:36.681020 14695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:36.681277 14695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:36.686811 14701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:36.686914 14695 server_base.cc:1034] running on GCE node
W20260808 07:01:36.686842 14704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:36.686811 14702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:36.687314 14695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:36.687637 14695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:36.688848 14695 hybrid_clock.cc:648] HybridClock initialized: now 1786172496688811 us; error 39 us; skew 500 ppm
Aug 08 07:01:36 dist-test-slave-8tdl krb5kdc[14682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172496, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:36.691769 14695 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:01:36.692307 14695 webserver.cc:460] Webserver started at http://127.13.207.62:43459/ using document root <none> and password file <none>
I20260808 07:01:36.692548 14695 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:36.692595 14695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:36.692773 14695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:36.693903 14695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "5efb2847f5fd44debee5ee96e4215054"
format_stamp: "Formatted at 2026-08-08 07:01:36 on dist-test-slave-8tdl"
server_key: "6a9385a11fd3cef8a25c973c08d0c89a"
server_key_iv: "47db2656686d4f7e60d7ff1f33d384fa"
server_key_version: "encryptionkey@0"
I20260808 07:01:36.694355 14695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "5efb2847f5fd44debee5ee96e4215054"
format_stamp: "Formatted at 2026-08-08 07:01:36 on dist-test-slave-8tdl"
server_key: "6a9385a11fd3cef8a25c973c08d0c89a"
server_key_iv: "47db2656686d4f7e60d7ff1f33d384fa"
server_key_version: "encryptionkey@0"
I20260808 07:01:36.695993 14695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:01:36.696907 14711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:36.697170 14695 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:36.697296 14695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "5efb2847f5fd44debee5ee96e4215054"
format_stamp: "Formatted at 2026-08-08 07:01:36 on dist-test-slave-8tdl"
server_key: "6a9385a11fd3cef8a25c973c08d0c89a"
server_key_iv: "47db2656686d4f7e60d7ff1f33d384fa"
server_key_version: "encryptionkey@0"
I20260808 07:01:36.697427 14695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:36.718993 14695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:36.721751 14695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:36.721942 14695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:36.727176 14695 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:41117
I20260808 07:01:36.727273 14764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:41117 every 8 connection(s)
I20260808 07:01:36.727802 14695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:01:36.728791 14765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:36.731467 14765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5efb2847f5fd44debee5ee96e4215054: Bootstrap starting.
I20260808 07:01:36.732120 14765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5efb2847f5fd44debee5ee96e4215054: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:36.732400 14765 log.cc:824] T 00000000000000000000000000000000 P 5efb2847f5fd44debee5ee96e4215054: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:36.733309 14765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5efb2847f5fd44debee5ee96e4215054: No bootstrap required, opened a new log
I20260808 07:01:36.734237 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 14695
I20260808 07:01:36.734330 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:01:36.734496 14140 external_mini_cluster.cc:1428] Setting key 40b9af8b35f9e4d28876bd1622fae2b0
I20260808 07:01:36.735569 14765 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5efb2847f5fd44debee5ee96e4215054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5efb2847f5fd44debee5ee96e4215054" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41117 } }
I20260808 07:01:36.735714 14765 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5efb2847f5fd44debee5ee96e4215054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:36.735780 14765 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5efb2847f5fd44debee5ee96e4215054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5efb2847f5fd44debee5ee96e4215054, State: Initialized, Role: FOLLOWER
I20260808 07:01:36.735929 14765 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5efb2847f5fd44debee5ee96e4215054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5efb2847f5fd44debee5ee96e4215054" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41117 } }
I20260808 07:01:36.736039 14765 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5efb2847f5fd44debee5ee96e4215054 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:01:36.736074 14765 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5efb2847f5fd44debee5ee96e4215054 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:01:36.736124 14765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5efb2847f5fd44debee5ee96e4215054 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:36.737389 14765 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5efb2847f5fd44debee5ee96e4215054 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5efb2847f5fd44debee5ee96e4215054" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41117 } }
I20260808 07:01:36.737491 14765 leader_election.cc:302] T 00000000000000000000000000000000 P 5efb2847f5fd44debee5ee96e4215054 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5efb2847f5fd44debee5ee96e4215054; no voters:
I20260808 07:01:36.737699 14765 leader_election.cc:288] T 00000000000000000000000000000000 P 5efb2847f5fd44debee5ee96e4215054 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:01:36.737749 14770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5efb2847f5fd44debee5ee96e4215054 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:36.737911 14770 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5efb2847f5fd44debee5ee96e4215054 [term 1 LEADER]: Becoming Leader. State: Replica: 5efb2847f5fd44debee5ee96e4215054, State: Running, Role: LEADER
I20260808 07:01:36.738000 14765 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5efb2847f5fd44debee5ee96e4215054 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:01:36.738011 14770 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5efb2847f5fd44debee5ee96e4215054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5efb2847f5fd44debee5ee96e4215054" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41117 } }
I20260808 07:01:36.738708 14772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5efb2847f5fd44debee5ee96e4215054 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5efb2847f5fd44debee5ee96e4215054. Latest consensus state: current_term: 1 leader_uuid: "5efb2847f5fd44debee5ee96e4215054" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5efb2847f5fd44debee5ee96e4215054" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41117 } } }
I20260808 07:01:36.738912 14772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5efb2847f5fd44debee5ee96e4215054 [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:36.738690 14771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5efb2847f5fd44debee5ee96e4215054 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5efb2847f5fd44debee5ee96e4215054" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5efb2847f5fd44debee5ee96e4215054" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41117 } } }
I20260808 07:01:36.739333 14781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:01:36.739428 14771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5efb2847f5fd44debee5ee96e4215054 [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:36.739948 14781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 07:01:36 dist-test-slave-8tdl krb5kdc[14682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:36.742408 14781 catalog_manager.cc:1347] Generated new cluster ID: 365a67720fbc4428b7b6d3426aebba97
I20260808 07:01:36.742465 14781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:01:36.746999 14769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:36.735180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33300 (local address 127.13.207.62:41117)
0808 07:01:36.735434 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:36.735443 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:36.735473 (+ 30us) server_negotiation.cc:407] Connection header received
0808 07:01:36.735589 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:36.735594 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:36.735640 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:36.735800 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:36.736667 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:36.737989 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:36.739190 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:36.739864 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:36.739977 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:36.742877 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:36.742882 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:36.742882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:36.742896 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:36.745132 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:36.745582 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:36.745585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:36.745586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:36.745740 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:36.745961 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:36.745962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:36.745962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:36.746313 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:36.746500 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:36.746578 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 07:01:36.746767 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":110,"threads_started":1}
I20260808 07:01:36.751197 14781 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:01:36.751885 14781 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:01:36.757959 14781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5efb2847f5fd44debee5ee96e4215054: Generated new TSK 0
I20260808 07:01:36.758185 14781 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
I20260808 07:01:36.799881 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:41117
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:37441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:36.889957 14791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:36.890246 14791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:36.890277 14791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:36.891767 14791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:36.891829 14791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:36.891865 14791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:01:36.893822 14791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:37441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:41117
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:36.894147 14791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:36.894423 14791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:36.900849 14798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:36.900852 14797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:36.901082 14791 server_base.cc:1034] running on GCE node
W20260808 07:01:36.900866 14800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:36.901425 14791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:36.901758 14791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:36.903047 14791 hybrid_clock.cc:648] HybridClock initialized: now 1786172496903018 us; error 47 us; skew 500 ppm
Aug 08 07:01:36 dist-test-slave-8tdl krb5kdc[14682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172496, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:36.906104 14791 init.cc:377] Logged in from keytab as kudu/127.13.207.1@KRBTEST.COM (short username kudu)
I20260808 07:01:36.906636 14791 webserver.cc:460] Webserver started at http://127.13.207.1:39571/ using document root <none> and password file <none>
I20260808 07:01:36.906924 14791 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:36.907002 14791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:36.907160 14791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:36.908308 14791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "886da17385f44367b76bcb479cd59958"
format_stamp: "Formatted at 2026-08-08 07:01:36 on dist-test-slave-8tdl"
server_key: "d80a4d074f66bfc8478d0084f65feb93"
server_key_iv: "6d8ceb2f24fe41fa00377fd619ea69f7"
server_key_version: "encryptionkey@0"
I20260808 07:01:36.908703 14791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "886da17385f44367b76bcb479cd59958"
format_stamp: "Formatted at 2026-08-08 07:01:36 on dist-test-slave-8tdl"
server_key: "d80a4d074f66bfc8478d0084f65feb93"
server_key_iv: "6d8ceb2f24fe41fa00377fd619ea69f7"
server_key_version: "encryptionkey@0"
I20260808 07:01:36.910248 14791 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:01:36.911556 14807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:36.911901 14791 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:36.912133 14791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "886da17385f44367b76bcb479cd59958"
format_stamp: "Formatted at 2026-08-08 07:01:36 on dist-test-slave-8tdl"
server_key: "d80a4d074f66bfc8478d0084f65feb93"
server_key_iv: "6d8ceb2f24fe41fa00377fd619ea69f7"
server_key_version: "encryptionkey@0"
I20260808 07:01:36.912223 14791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:36.925824 14791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:36.928498 14791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:36.928637 14791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:36.928854 14791 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:36.929221 14791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:36.929292 14791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:36.929338 14791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:36.929373 14791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:36.936487 14791 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:35831
I20260808 07:01:36.936648 14920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:35831 every 8 connection(s)
I20260808 07:01:36.936965 14791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:01:36 dist-test-slave-8tdl krb5kdc[14682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172496, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:36.946964 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 14791
I20260808 07:01:36.947083 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:01:36.947260 14140 external_mini_cluster.cc:1428] Setting key f220672d654c95e26da72aaedc75c1b9
I20260808 07:01:36.948644 14769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:36.937924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:59509 (local address 127.13.207.62:41117)
0808 07:01:36.938071 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:36.938073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:36.938274 (+ 201us) server_negotiation.cc:407] Connection header received
0808 07:01:36.939417 (+ 1143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:36.939418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:36.939463 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:36.939530 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:36.940217 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:36.940745 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:36.942006 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:36.942226 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:36.942290 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:36.944916 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:36.944918 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:36.944918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:36.944935 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:36.946809 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:36.947432 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:36.947434 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:36.947434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:36.947507 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:36.947847 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:36.947848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:36.947848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:36.947986 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:36.948069 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:36.948433 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 07:01:36.948541 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260808 07:01:36.948941 14923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:36.937920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:41117 (local address 127.13.207.1:59509)
0808 07:01:36.938104 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:36.938151 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:36.939309 (+ 1158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:36.939615 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:36.939619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:36.939893 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:36.940100 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:36.940104 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:36.941005 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:36.941006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:36.941853 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:36.941855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:36.942357 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:36.942359 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:36.942484 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:36.943167 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:36.943177 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:36.944794 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:36.946919 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:36.946932 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:36.946934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:36.947295 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:36.947588 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:36.947589 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:36.947589 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:36.947767 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:36.948087 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:36.948095 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:36.948259 (+ 164us) client_negotiation.cc:769] Sending connection context
0808 07:01:36.948382 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 07:01:36.948598 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":81,"threads_started":1}
I20260808 07:01:36.949546 14921 heartbeater.cc:347] Connected to a master server at 127.13.207.62:41117
I20260808 07:01:36.949680 14921 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:36.949910 14921 heartbeater.cc:511] Master 127.13.207.62:41117 requested a full tablet report, sending...
I20260808 07:01:36.950629 14728 ts_manager.cc:194] Registered new tserver with Master: 886da17385f44367b76bcb479cd59958 (127.13.207.1:35831)
I20260808 07:01:36.951687 14728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:59509
WARNING: no policy specified for kudu/127.13.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
I20260808 07:01:36.982800 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:41117
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:37441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:37.076165 14926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:37.076428 14926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:37.076457 14926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:37.077775 14926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:37.077836 14926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:37.077867 14926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:01:37.079523 14926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:37441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:41117
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:37.079754 14926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:37.079962 14926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:37.085672 14934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:37.085712 14926 server_base.cc:1034] running on GCE node
W20260808 07:01:37.085672 14936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:37.085712 14933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:37.086283 14926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:37.086632 14926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:37.087870 14926 hybrid_clock.cc:648] HybridClock initialized: now 1786172497087848 us; error 36 us; skew 500 ppm
Aug 08 07:01:37 dist-test-slave-8tdl krb5kdc[14682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172497, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:37.090898 14926 init.cc:377] Logged in from keytab as kudu/127.13.207.2@KRBTEST.COM (short username kudu)
I20260808 07:01:37.091431 14926 webserver.cc:460] Webserver started at http://127.13.207.2:42643/ using document root <none> and password file <none>
I20260808 07:01:37.091696 14926 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:37.091744 14926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:37.091908 14926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:37.093052 14926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "ea0ac5c6cbee4903a7131dfabd74e58b"
format_stamp: "Formatted at 2026-08-08 07:01:37 on dist-test-slave-8tdl"
server_key: "3be03356aa20cbed16d9af635389b4bd"
server_key_iv: "35b89103c78e796d76ceea04cb964581"
server_key_version: "encryptionkey@0"
I20260808 07:01:37.093463 14926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "ea0ac5c6cbee4903a7131dfabd74e58b"
format_stamp: "Formatted at 2026-08-08 07:01:37 on dist-test-slave-8tdl"
server_key: "3be03356aa20cbed16d9af635389b4bd"
server_key_iv: "35b89103c78e796d76ceea04cb964581"
server_key_version: "encryptionkey@0"
I20260808 07:01:37.095175 14926 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 07:01:37.096167 14943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:37.096429 14926 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:37.096541 14926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "ea0ac5c6cbee4903a7131dfabd74e58b"
format_stamp: "Formatted at 2026-08-08 07:01:37 on dist-test-slave-8tdl"
server_key: "3be03356aa20cbed16d9af635389b4bd"
server_key_iv: "35b89103c78e796d76ceea04cb964581"
server_key_version: "encryptionkey@0"
I20260808 07:01:37.096637 14926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:37.106238 14926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:37.108893 14926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:37.109238 14926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:37.109536 14926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:37.109908 14926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:37.110050 14926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:37.110108 14926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:37.110136 14926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:37.118574 14926 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:40113
I20260808 07:01:37.118597 15057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:40113 every 8 connection(s)
I20260808 07:01:37.119052 14926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:01:37.120187 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 14926
I20260808 07:01:37.120410 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:01:37.120787 14140 external_mini_cluster.cc:1428] Setting key 11ca197c800ae1c73cf3854979a39e97
Aug 08 07:01:37 dist-test-slave-8tdl krb5kdc[14682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172497, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.207.3@KRBTEST.COM; defaulting to no policy
I20260808 07:01:37.132762 14769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:37.120160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:32991 (local address 127.13.207.62:41117)
0808 07:01:37.120232 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:37.120234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:37.121210 (+ 976us) server_negotiation.cc:407] Connection header received
0808 07:01:37.122393 (+ 1183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:37.122395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:37.122409 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:37.122457 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:37.123273 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.123943 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.125414 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.125716 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.125799 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:37.128393 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:37.128395 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:37.128395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:37.128405 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:37.130473 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:37.131280 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:37.131282 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:37.131283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:37.131368 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:37.131852 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:37.131853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:37.131853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:37.132040 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:37.132122 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:37.132589 (+ 467us) server_negotiation.cc:299] Negotiation successful
0808 07:01:37.132692 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:01:37.132882 15060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:37.119946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:41117 (local address 127.13.207.2:32991)
0808 07:01:37.121036 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:37.121095 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:37.122274 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:37.122542 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:37.122551 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:37.122935 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:37.123151 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:37.123155 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.124082 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.124083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:37.125284 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:37.125287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.125829 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.125830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:37.125976 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:37.126662 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:37.126715 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:37.128267 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:37.130598 (+ 2331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:37.130607 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:37.130609 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:37.131074 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:37.131615 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:37.131616 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:37.131616 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:37.131753 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:37.132160 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:37.132169 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:37.132377 (+ 208us) client_negotiation.cc:769] Sending connection context
0808 07:01:37.132491 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 07:01:37.132644 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1035,"thread_start_us":106,"threads_started":1}
I20260808 07:01:37.133518 15058 heartbeater.cc:347] Connected to a master server at 127.13.207.62:41117
I20260808 07:01:37.133616 15058 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:37.133823 15058 heartbeater.cc:511] Master 127.13.207.62:41117 requested a full tablet report, sending...
I20260808 07:01:37.134270 14728 ts_manager.cc:194] Registered new tserver with Master: ea0ac5c6cbee4903a7131dfabd74e58b (127.13.207.2:40113)
I20260808 07:01:37.134770 14728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:32991
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
I20260808 07:01:37.160647 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:41117
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:37441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:37.256055 15063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:37.256349 15063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:37.256392 15063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:37.257721 15063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:37.257805 15063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:37.257856 15063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:01:37.260113 15063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:37441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:41117
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:37.260476 15063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:37.260867 15063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:37.267246 15069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:37.267246 15070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:37.267321 15072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:37.267658 15063 server_base.cc:1034] running on GCE node
I20260808 07:01:37.267941 15063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:37.268332 15063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:37.269580 15063 hybrid_clock.cc:648] HybridClock initialized: now 1786172497269557 us; error 49 us; skew 500 ppm
Aug 08 07:01:37 dist-test-slave-8tdl krb5kdc[14682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172497, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:37.273136 15063 init.cc:377] Logged in from keytab as kudu/127.13.207.3@KRBTEST.COM (short username kudu)
I20260808 07:01:37.273769 15063 webserver.cc:460] Webserver started at http://127.13.207.3:45601/ using document root <none> and password file <none>
I20260808 07:01:37.274060 15063 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:37.274135 15063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:37.274292 15063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:37.275609 15063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "d01fa99407864ae483936cb27043e470"
format_stamp: "Formatted at 2026-08-08 07:01:37 on dist-test-slave-8tdl"
server_key: "28fa789006f6f737c89666f225490bcc"
server_key_iv: "981c301544d3ff4517710eb386c9dab2"
server_key_version: "encryptionkey@0"
I20260808 07:01:37.276067 15063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "d01fa99407864ae483936cb27043e470"
format_stamp: "Formatted at 2026-08-08 07:01:37 on dist-test-slave-8tdl"
server_key: "28fa789006f6f737c89666f225490bcc"
server_key_iv: "981c301544d3ff4517710eb386c9dab2"
server_key_version: "encryptionkey@0"
I20260808 07:01:37.277683 15063 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:01:37.278621 15079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:37.278893 15063 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:37.279016 15063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "d01fa99407864ae483936cb27043e470"
format_stamp: "Formatted at 2026-08-08 07:01:37 on dist-test-slave-8tdl"
server_key: "28fa789006f6f737c89666f225490bcc"
server_key_iv: "981c301544d3ff4517710eb386c9dab2"
server_key_version: "encryptionkey@0"
I20260808 07:01:37.279122 15063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:37.289299 15063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:37.292753 15063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:37.292954 15063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:37.293231 15063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:37.293567 15063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:37.293603 15063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:37.293629 15063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:37.293644 15063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:37.302313 15063 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:35547
I20260808 07:01:37.302351 15193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:35547 every 8 connection(s)
I20260808 07:01:37.302718 15063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:01:37.308152 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 15063
I20260808 07:01:37.308290 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:01:37.308581 14140 external_mini_cluster.cc:1428] Setting key 02d052ba2cdcdd1de2bc4cd80f6321e6
Aug 08 07:01:37 dist-test-slave-8tdl krb5kdc[14682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172497, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:37.314247 14769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:37.303621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:45733 (local address 127.13.207.62:41117)
0808 07:01:37.303707 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:37.303710 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:37.303830 (+ 120us) server_negotiation.cc:407] Connection header received
0808 07:01:37.304866 (+ 1036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:37.304868 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:37.304881 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:37.304956 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:37.305757 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.306420 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.307656 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.307863 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.307993 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:37.311058 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:37.311060 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:37.311060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:37.311068 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:37.312664 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:37.313144 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:37.313145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:37.313145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:37.313204 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:37.313506 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:37.313507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:37.313507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:37.313626 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:37.313681 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:37.314090 (+ 409us) server_negotiation.cc:299] Negotiation successful
0808 07:01:37.314169 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 07:01:37.314371 15196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:37.303519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:41117 (local address 127.13.207.3:45733)
0808 07:01:37.303694 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:37.303734 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:37.304735 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:37.305055 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:37.305060 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:37.305371 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:37.305590 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:37.305594 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.306542 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.306544 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:37.307536 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:37.307539 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.308052 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.308054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:37.308251 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:37.308975 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:37.308985 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:37.310929 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:37.312775 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:37.312782 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:37.312785 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:37.313061 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:37.313295 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:37.313297 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:37.313297 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:37.313438 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:37.313693 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:37.313698 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:37.313833 (+ 135us) client_negotiation.cc:769] Sending connection context
0808 07:01:37.313987 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 07:01:37.314137 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":77,"threads_started":1}
I20260808 07:01:37.314989 15194 heartbeater.cc:347] Connected to a master server at 127.13.207.62:41117
I20260808 07:01:37.315101 15194 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:37.315354 15194 heartbeater.cc:511] Master 127.13.207.62:41117 requested a full tablet report, sending...
I20260808 07:01:37.315829 14728 ts_manager.cc:194] Registered new tserver with Master: d01fa99407864ae483936cb27043e470 (127.13.207.3:35547)
I20260808 07:01:37.316386 14728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:45733
I20260808 07:01:37.321394 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:01:37.331189 14769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:37.323296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33308 (local address 127.13.207.62:41117)
0808 07:01:37.323358 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:37.323361 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:37.323413 (+ 52us) server_negotiation.cc:407] Connection header received
0808 07:01:37.323490 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:37.323490 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:37.323499 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:37.323549 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:37.324138 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.324666 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.325584 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.325807 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.325941 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:37.327520 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:37.327521 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:37.327522 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:37.327530 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:37.329170 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:37.329674 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:37.329676 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:37.329676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:37.329740 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:37.330157 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:37.330158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:37.330159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:37.330342 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:37.330441 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:37.330837 (+ 396us) server_negotiation.cc:299] Negotiation successful
0808 07:01:37.331043 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:01:37.333947 14728 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:01:37.335695 14728 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:01:37.343545 15208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:37.338602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:35547 (local address 127.0.0.1:38466)
0808 07:01:37.338840 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:37.338882 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:37.338962 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:37.339926 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:37.339927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:37.339928 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:37.339985 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:37.339986 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.341245 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.341246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:37.342403 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:37.342405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.343260 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.343261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:37.343311 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:37.343316 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:01:37.343370 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 07:01:37.343440 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":75,"threads_started":1}
I20260808 07:01:37.343698 15207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:37.338605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:35831 (local address 127.0.0.1:60540)
0808 07:01:37.339068 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:37.339075 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:37.339128 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:37.339875 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:37.339876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:37.339878 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:37.339928 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:37.339930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.340882 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.340884 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:37.342328 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:37.342331 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.343358 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.343360 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:37.343428 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:37.343433 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:01:37.343494 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 07:01:37.343544 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":2,"thread_start_us":139,"threads_started":1}
I20260808 07:01:37.343681 15211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:37.339435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60540 (local address 127.13.207.1:35831)
0808 07:01:37.339696 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:37.339700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:37.339710 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:01:37.339748 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:37.339751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:37.339766 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:37.340014 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:37.340102 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.340767 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.342452 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.343222 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.343350 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:37.343413 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:37.343517 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 07:01:37.343561 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":123,"threads_started":1}
I20260808 07:01:37.343868 15206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:37.338605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:40113 (local address 127.0.0.1:44402)
0808 07:01:37.339054 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:37.339066 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:37.339156 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:37.339488 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:37.339492 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:37.339495 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:37.339600 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:37.339603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.341066 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.341067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:37.342182 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:37.342185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.343206 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.343207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:37.343298 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:37.343316 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 07:01:37.343391 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 07:01:37.343440 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"mutex_wait_us":40,"thread_start_us":137,"threads_started":1}
I20260808 07:01:37.344278 15209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:37.338837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38466 (local address 127.13.207.3:35547)
0808 07:01:37.339725 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:37.339730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:37.339740 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:01:37.339780 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:37.339782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:37.339800 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:37.339928 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:37.340086 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.341119 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.342497 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.343138 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.344026 (+ 888us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:37.344102 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:37.344133 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 07:01:37.344190 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":840,"thread_start_us":115,"threads_started":1}
I20260808 07:01:37.344264 15210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:37.339047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44402 (local address 127.13.207.2:40113)
0808 07:01:37.339269 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:37.339274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:37.339304 (+ 30us) server_negotiation.cc:407] Connection header received
0808 07:01:37.339349 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:37.339352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:37.339369 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:37.339441 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:37.339723 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.340936 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.342280 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.343099 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.343185 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:37.343250 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:37.343497 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 07:01:37.343543 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":84,"threads_started":1}
I20260808 07:01:37.344522 14855 tablet_service.cc:1467] Processing CreateTablet for tablet 0ab9b4780101477895aa9a247b79bd31 (DEFAULT_TABLE table=test-table [id=dc730ad137eb477aaf2274e2dc7a7cc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:37.344938 14855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ab9b4780101477895aa9a247b79bd31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:37.344899 15127 tablet_service.cc:1467] Processing CreateTablet for tablet 0ab9b4780101477895aa9a247b79bd31 (DEFAULT_TABLE table=test-table [id=dc730ad137eb477aaf2274e2dc7a7cc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:37.344905 14992 tablet_service.cc:1467] Processing CreateTablet for tablet 0ab9b4780101477895aa9a247b79bd31 (DEFAULT_TABLE table=test-table [id=dc730ad137eb477aaf2274e2dc7a7cc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:37.345217 14992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ab9b4780101477895aa9a247b79bd31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:37.345217 15127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ab9b4780101477895aa9a247b79bd31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:37.348161 15212 tablet_bootstrap.cc:492] T 0ab9b4780101477895aa9a247b79bd31 P 886da17385f44367b76bcb479cd59958: Bootstrap starting.
I20260808 07:01:37.348733 15213 tablet_bootstrap.cc:492] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470: Bootstrap starting.
I20260808 07:01:37.349476 15213 tablet_bootstrap.cc:654] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:37.350198 15213 log.cc:824] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:37.350630 15212 tablet_bootstrap.cc:654] T 0ab9b4780101477895aa9a247b79bd31 P 886da17385f44367b76bcb479cd59958: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:37.350636 15214 tablet_bootstrap.cc:492] T 0ab9b4780101477895aa9a247b79bd31 P ea0ac5c6cbee4903a7131dfabd74e58b: Bootstrap starting.
I20260808 07:01:37.351183 15213 tablet_bootstrap.cc:492] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470: No bootstrap required, opened a new log
I20260808 07:01:37.351260 15213 ts_tablet_manager.cc:1397] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:01:37.351424 15214 tablet_bootstrap.cc:654] T 0ab9b4780101477895aa9a247b79bd31 P ea0ac5c6cbee4903a7131dfabd74e58b: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:37.351722 15212 log.cc:824] T 0ab9b4780101477895aa9a247b79bd31 P 886da17385f44367b76bcb479cd59958: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:37.352234 15214 log.cc:824] T 0ab9b4780101477895aa9a247b79bd31 P ea0ac5c6cbee4903a7131dfabd74e58b: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:37.352697 15212 tablet_bootstrap.cc:492] T 0ab9b4780101477895aa9a247b79bd31 P 886da17385f44367b76bcb479cd59958: No bootstrap required, opened a new log
I20260808 07:01:37.352780 15212 ts_tablet_manager.cc:1397] T 0ab9b4780101477895aa9a247b79bd31 P 886da17385f44367b76bcb479cd59958: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260808 07:01:37.353003 15214 tablet_bootstrap.cc:492] T 0ab9b4780101477895aa9a247b79bd31 P ea0ac5c6cbee4903a7131dfabd74e58b: No bootstrap required, opened a new log
I20260808 07:01:37.353085 15214 ts_tablet_manager.cc:1397] T 0ab9b4780101477895aa9a247b79bd31 P ea0ac5c6cbee4903a7131dfabd74e58b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:01:37.353305 15213 raft_consensus.cc:348] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886da17385f44367b76bcb479cd59958" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 35831 } } peers { permanent_uuid: "ea0ac5c6cbee4903a7131dfabd74e58b" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40113 } } peers { permanent_uuid: "d01fa99407864ae483936cb27043e470" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 35547 } }
I20260808 07:01:37.353510 15213 raft_consensus.cc:374] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:37.353549 15213 raft_consensus.cc:729] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d01fa99407864ae483936cb27043e470, State: Initialized, Role: FOLLOWER
I20260808 07:01:37.353708 15213 consensus_queue.cc:260] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886da17385f44367b76bcb479cd59958" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 35831 } } peers { permanent_uuid: "ea0ac5c6cbee4903a7131dfabd74e58b" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40113 } } peers { permanent_uuid: "d01fa99407864ae483936cb27043e470" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 35547 } }
I20260808 07:01:37.354031 15213 ts_tablet_manager.cc:1428] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:01:37.354074 15194 heartbeater.cc:503] Master 127.13.207.62:41117 was elected leader, sending a full tablet report...
I20260808 07:01:37.355444 15212 raft_consensus.cc:348] T 0ab9b4780101477895aa9a247b79bd31 P 886da17385f44367b76bcb479cd59958 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886da17385f44367b76bcb479cd59958" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 35831 } } peers { permanent_uuid: "ea0ac5c6cbee4903a7131dfabd74e58b" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40113 } } peers { permanent_uuid: "d01fa99407864ae483936cb27043e470" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 35547 } }
I20260808 07:01:37.355667 15212 raft_consensus.cc:374] T 0ab9b4780101477895aa9a247b79bd31 P 886da17385f44367b76bcb479cd59958 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:37.355723 15212 raft_consensus.cc:729] T 0ab9b4780101477895aa9a247b79bd31 P 886da17385f44367b76bcb479cd59958 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 886da17385f44367b76bcb479cd59958, State: Initialized, Role: FOLLOWER
I20260808 07:01:37.355856 15212 consensus_queue.cc:260] T 0ab9b4780101477895aa9a247b79bd31 P 886da17385f44367b76bcb479cd59958 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886da17385f44367b76bcb479cd59958" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 35831 } } peers { permanent_uuid: "ea0ac5c6cbee4903a7131dfabd74e58b" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40113 } } peers { permanent_uuid: "d01fa99407864ae483936cb27043e470" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 35547 } }
I20260808 07:01:37.355930 15214 raft_consensus.cc:348] T 0ab9b4780101477895aa9a247b79bd31 P ea0ac5c6cbee4903a7131dfabd74e58b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886da17385f44367b76bcb479cd59958" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 35831 } } peers { permanent_uuid: "ea0ac5c6cbee4903a7131dfabd74e58b" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40113 } } peers { permanent_uuid: "d01fa99407864ae483936cb27043e470" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 35547 } }
I20260808 07:01:37.356160 15214 raft_consensus.cc:374] T 0ab9b4780101477895aa9a247b79bd31 P ea0ac5c6cbee4903a7131dfabd74e58b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:37.356210 15212 ts_tablet_manager.cc:1428] T 0ab9b4780101477895aa9a247b79bd31 P 886da17385f44367b76bcb479cd59958: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
I20260808 07:01:37.356215 14921 heartbeater.cc:503] Master 127.13.207.62:41117 was elected leader, sending a full tablet report...
I20260808 07:01:37.356273 15214 raft_consensus.cc:729] T 0ab9b4780101477895aa9a247b79bd31 P ea0ac5c6cbee4903a7131dfabd74e58b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea0ac5c6cbee4903a7131dfabd74e58b, State: Initialized, Role: FOLLOWER
I20260808 07:01:37.356359 15214 consensus_queue.cc:260] T 0ab9b4780101477895aa9a247b79bd31 P ea0ac5c6cbee4903a7131dfabd74e58b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886da17385f44367b76bcb479cd59958" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 35831 } } peers { permanent_uuid: "ea0ac5c6cbee4903a7131dfabd74e58b" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40113 } } peers { permanent_uuid: "d01fa99407864ae483936cb27043e470" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 35547 } }
I20260808 07:01:37.356635 15214 ts_tablet_manager.cc:1428] T 0ab9b4780101477895aa9a247b79bd31 P ea0ac5c6cbee4903a7131dfabd74e58b: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 07:01:37.356649 15058 heartbeater.cc:503] Master 127.13.207.62:41117 was elected leader, sending a full tablet report...
W20260808 07:01:37.369800 15059 tablet.cc:2369] T 0ab9b4780101477895aa9a247b79bd31 P ea0ac5c6cbee4903a7131dfabd74e58b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:01:37.437815 14922 tablet.cc:2369] T 0ab9b4780101477895aa9a247b79bd31 P 886da17385f44367b76bcb479cd59958: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:01:37.553376 15195 tablet.cc:2369] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:01:37.615216 15218 raft_consensus.cc:482] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:01:37.615386 15218 raft_consensus.cc:504] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886da17385f44367b76bcb479cd59958" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 35831 } } peers { permanent_uuid: "ea0ac5c6cbee4903a7131dfabd74e58b" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40113 } } peers { permanent_uuid: "d01fa99407864ae483936cb27043e470" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 35547 } }
I20260808 07:01:37.615761 15218 leader_election.cc:288] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 886da17385f44367b76bcb479cd59958 (127.13.207.1:35831), ea0ac5c6cbee4903a7131dfabd74e58b (127.13.207.2:40113)
I20260808 07:01:37.619318 15210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:37.616366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:45073 (local address 127.13.207.2:40113)
0808 07:01:37.616417 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:37.616419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:37.616427 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:01:37.616454 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:37.616455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:37.616464 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:37.616525 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:37.616830 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.617324 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.618441 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.618910 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.618989 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:37.619001 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:37.619185 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 07:01:37.619238 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:01:37.619333 15221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:37.616021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:40113 (local address 127.13.207.3:45073)
0808 07:01:37.616298 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:37.616326 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:37.616380 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:37.616571 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:37.616572 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:37.616574 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:37.616716 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:37.616718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.617412 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.617413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:37.618347 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:37.618350 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.619025 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.619027 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:37.619108 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:37.619112 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:01:37.619169 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 07:01:37.619224 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":17,"thread_start_us":153,"threads_started":1}
I20260808 07:01:37.619354 15211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:37.616071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:42183 (local address 127.13.207.1:35831)
0808 07:01:37.616129 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:37.616132 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:37.616274 (+ 142us) server_negotiation.cc:407] Connection header received
0808 07:01:37.616312 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:37.616313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:37.616330 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:37.616395 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:37.616719 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.617354 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.618500 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.618915 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.619045 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:37.619055 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:37.619244 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 07:01:37.619287 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 07:01:37.619338 15196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:37.616004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:35831 (local address 127.13.207.3:42183)
0808 07:01:37.616062 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:37.616076 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:37.616229 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:37.616485 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:37.616487 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:37.616489 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:37.616569 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:37.616571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.617480 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.617481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:37.618375 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:37.618378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.619058 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.619059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:37.619156 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:37.619161 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:01:37.619226 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 07:01:37.619264 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":49}
I20260808 07:01:37.619685 15012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ab9b4780101477895aa9a247b79bd31" candidate_uuid: "d01fa99407864ae483936cb27043e470" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea0ac5c6cbee4903a7131dfabd74e58b" is_pre_election: true
I20260808 07:01:37.619868 15012 raft_consensus.cc:2454] T 0ab9b4780101477895aa9a247b79bd31 P ea0ac5c6cbee4903a7131dfabd74e58b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d01fa99407864ae483936cb27043e470 in term 0.
I20260808 07:01:37.619846 14875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ab9b4780101477895aa9a247b79bd31" candidate_uuid: "d01fa99407864ae483936cb27043e470" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "886da17385f44367b76bcb479cd59958" is_pre_election: true
I20260808 07:01:37.620048 14875 raft_consensus.cc:2454] T 0ab9b4780101477895aa9a247b79bd31 P 886da17385f44367b76bcb479cd59958 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d01fa99407864ae483936cb27043e470 in term 0.
I20260808 07:01:37.620201 15083 leader_election.cc:302] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d01fa99407864ae483936cb27043e470, ea0ac5c6cbee4903a7131dfabd74e58b; no voters:
I20260808 07:01:37.620371 15218 raft_consensus.cc:2781] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:01:37.620447 15218 raft_consensus.cc:482] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:01:37.620486 15218 raft_consensus.cc:3037] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:37.621233 15218 raft_consensus.cc:504] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886da17385f44367b76bcb479cd59958" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 35831 } } peers { permanent_uuid: "ea0ac5c6cbee4903a7131dfabd74e58b" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40113 } } peers { permanent_uuid: "d01fa99407864ae483936cb27043e470" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 35547 } }
I20260808 07:01:37.621421 15218 leader_election.cc:288] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470 [CANDIDATE]: Term 1 election: Requested vote from peers 886da17385f44367b76bcb479cd59958 (127.13.207.1:35831), ea0ac5c6cbee4903a7131dfabd74e58b (127.13.207.2:40113)
I20260808 07:01:37.621634 14875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ab9b4780101477895aa9a247b79bd31" candidate_uuid: "d01fa99407864ae483936cb27043e470" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "886da17385f44367b76bcb479cd59958"
I20260808 07:01:37.621683 15012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ab9b4780101477895aa9a247b79bd31" candidate_uuid: "d01fa99407864ae483936cb27043e470" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea0ac5c6cbee4903a7131dfabd74e58b"
I20260808 07:01:37.621738 14875 raft_consensus.cc:3037] T 0ab9b4780101477895aa9a247b79bd31 P 886da17385f44367b76bcb479cd59958 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:37.621758 15012 raft_consensus.cc:3037] T 0ab9b4780101477895aa9a247b79bd31 P ea0ac5c6cbee4903a7131dfabd74e58b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:37.622710 14875 raft_consensus.cc:2454] T 0ab9b4780101477895aa9a247b79bd31 P 886da17385f44367b76bcb479cd59958 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d01fa99407864ae483936cb27043e470 in term 1.
I20260808 07:01:37.622705 15012 raft_consensus.cc:2454] T 0ab9b4780101477895aa9a247b79bd31 P ea0ac5c6cbee4903a7131dfabd74e58b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d01fa99407864ae483936cb27043e470 in term 1.
I20260808 07:01:37.623006 15081 leader_election.cc:302] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 886da17385f44367b76bcb479cd59958, d01fa99407864ae483936cb27043e470; no voters:
I20260808 07:01:37.623138 15218 raft_consensus.cc:2781] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:37.623327 15218 raft_consensus.cc:686] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470 [term 1 LEADER]: Becoming Leader. State: Replica: d01fa99407864ae483936cb27043e470, State: Running, Role: LEADER
I20260808 07:01:37.623449 15218 consensus_queue.cc:237] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886da17385f44367b76bcb479cd59958" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 35831 } } peers { permanent_uuid: "ea0ac5c6cbee4903a7131dfabd74e58b" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40113 } } peers { permanent_uuid: "d01fa99407864ae483936cb27043e470" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 35547 } }
I20260808 07:01:37.624589 14727 catalog_manager.cc:5495] T 0ab9b4780101477895aa9a247b79bd31 P d01fa99407864ae483936cb27043e470 reported cstate change: term changed from 0 to 1, leader changed from <none> to d01fa99407864ae483936cb27043e470 (127.13.207.3). New cstate: current_term: 1 leader_uuid: "d01fa99407864ae483936cb27043e470" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "886da17385f44367b76bcb479cd59958" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 35831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea0ac5c6cbee4903a7131dfabd74e58b" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d01fa99407864ae483936cb27043e470" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 35547 } health_report { overall_health: HEALTHY } } }
Aug 08 07:01:37 dist-test-slave-8tdl krb5kdc[14682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.1@KRBTEST.COM
I20260808 07:01:37.706662 15211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:37.695676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60552 (local address 127.13.207.1:35831)
0808 07:01:37.695758 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:37.695761 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:37.695958 (+ 197us) server_negotiation.cc:407] Connection header received
0808 07:01:37.695998 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:37.695999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:37.696010 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:37.696063 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:37.697051 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.697667 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.698766 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.699324 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.699404 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:37.702879 (+ 3475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:37.702896 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:37.702897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:37.702906 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:37.704987 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:37.705441 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:37.705444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:37.705445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:37.705527 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:37.705814 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:37.705816 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:37.705816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:37.706005 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:37.706138 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:37.706388 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 07:01:37.706527 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 07:01:37 dist-test-slave-8tdl krb5kdc[14682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172497, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:01:37 dist-test-slave-8tdl krb5kdc[14682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172497, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.207.1@KRBTEST.COM
I20260808 07:01:37.739490 15211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:37.729439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60556 (local address 127.13.207.1:35831)
0808 07:01:37.729517 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:37.729520 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:37.730012 (+ 492us) server_negotiation.cc:407] Connection header received
0808 07:01:37.730102 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:37.730103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:37.730116 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:37.730171 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:37.731260 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.731885 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.732645 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.732863 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.732946 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:37.735808 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:37.735811 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:37.735811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:37.735821 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:37.737601 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:37.738283 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:37.738285 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:37.738285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:37.738357 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:37.738578 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:37.738579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:37.738580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:37.738760 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:37.738848 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:37.739245 (+ 397us) server_negotiation.cc:299] Negotiation successful
0808 07:01:37.739340 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
Aug 08 07:01:37 dist-test-slave-8tdl krb5kdc[14682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:01:37 dist-test-slave-8tdl krb5kdc[14682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.1@KRBTEST.COM
I20260808 07:01:37.769752 15211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:37.760505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60570 (local address 127.13.207.1:35831)
0808 07:01:37.760621 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:37.760623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:37.760635 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:01:37.760742 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:37.760743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:37.760756 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:37.760829 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:37.761557 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.762102 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.763198 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:37.763424 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:37.763523 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:37.766478 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:37.766480 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:37.766481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:37.766489 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:37.768093 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:37.768628 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:37.768630 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:37.768630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:37.768701 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:37.769069 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:37.769070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:37.769070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:37.769206 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:37.769311 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:37.769590 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 07:01:37.769669 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260808 07:01:37.771183 14835 tablet_service.cc:2908] Rejecting scan request for tablet 0ab9b4780101477895aa9a247b79bd31: Uninitialized: clean time has not yet been initialized
I20260808 07:01:37.772619 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 14791
I20260808 07:01:37.780570 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 14926
I20260808 07:01:37.788210 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 15063
I20260808 07:01:37.797492 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 14695
2026-08-08T07:01:37Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2507 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:01:37 dist-test-slave-8tdl krb5kdc[15236](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:01:37 dist-test-slave-8tdl krb5kdc[15236](info): set up 2 sockets
Aug 08 07:01:37 dist-test-slave-8tdl krb5kdc[15236](info): commencing operation
krb5kdc: starting...
W20260808 07:01:38.995826 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:01:39 dist-test-slave-8tdl krb5kdc[15236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:01:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:01:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:01:39.159708 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:40015
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:39959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:40015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:39.257395 15250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:39.257707 15250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:39.257754 15250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:39.259140 15250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:01:39.259230 15250 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 07:01:39.259254 15250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:39.259279 15250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:39.259311 15250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:01:39.259335 15250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:01:39.261090 15250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:39959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:40015
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:40015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:39.261440 15250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:39.261703 15250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:39.267081 15258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:39.267197 15250 server_base.cc:1034] running on GCE node
W20260808 07:01:39.267081 15256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:39.268311 15255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:39.268697 15250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:39.269204 15250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:39.270467 15250 hybrid_clock.cc:648] HybridClock initialized: now 1786172499270405 us; error 80 us; skew 500 ppm
Aug 08 07:01:39 dist-test-slave-8tdl krb5kdc[15236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172499, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:39.277745 15250 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:01:39.278359 15250 webserver.cc:460] Webserver started at http://127.13.207.62:40857/ using document root <none> and password file <none>
I20260808 07:01:39.278601 15250 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:39.278650 15250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:39.278855 15250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:39.280097 15250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "fbdd7ddae2494eff8bd05f069aae8f27"
format_stamp: "Formatted at 2026-08-08 07:01:39 on dist-test-slave-8tdl"
server_key: "1a5832681d7e1fbf75b1d8a5a836f0fe"
server_key_iv: "bfee10a4dd3be5b9b5787f3a4baca2e6"
server_key_version: "encryptionkey@0"
I20260808 07:01:39.280468 15250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "fbdd7ddae2494eff8bd05f069aae8f27"
format_stamp: "Formatted at 2026-08-08 07:01:39 on dist-test-slave-8tdl"
server_key: "1a5832681d7e1fbf75b1d8a5a836f0fe"
server_key_iv: "bfee10a4dd3be5b9b5787f3a4baca2e6"
server_key_version: "encryptionkey@0"
I20260808 07:01:39.282198 15250 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 07:01:39.283311 15265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:39.283586 15250 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:01:39.283691 15250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "fbdd7ddae2494eff8bd05f069aae8f27"
format_stamp: "Formatted at 2026-08-08 07:01:39 on dist-test-slave-8tdl"
server_key: "1a5832681d7e1fbf75b1d8a5a836f0fe"
server_key_iv: "bfee10a4dd3be5b9b5787f3a4baca2e6"
server_key_version: "encryptionkey@0"
I20260808 07:01:39.283763 15250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:39.296721 15250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:39.300423 15250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:39.300559 15250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:39.306355 15250 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:40015
I20260808 07:01:39.306546 15328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:40015 every 8 connection(s)
I20260808 07:01:39.306869 15250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:01:39.307273 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 15250
I20260808 07:01:39.307466 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:01:39.307690 14140 external_mini_cluster.cc:1428] Setting key 30721842375435955f9bf28f821cdad4
I20260808 07:01:39.307760 15329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:39.312001 15329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27: Bootstrap starting.
I20260808 07:01:39.312855 15329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:39.313195 15329 log.cc:824] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:39.314145 15329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27: No bootstrap required, opened a new log
Aug 08 07:01:39 dist-test-slave-8tdl krb5kdc[15236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:39.316808 15329 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbdd7ddae2494eff8bd05f069aae8f27" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40015 } }
I20260808 07:01:39.317062 15329 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:39.317119 15329 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbdd7ddae2494eff8bd05f069aae8f27, State: Initialized, Role: FOLLOWER
I20260808 07:01:39.317320 15329 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbdd7ddae2494eff8bd05f069aae8f27" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40015 } }
I20260808 07:01:39.317430 15329 raft_consensus.cc:388] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:01:39.317457 15329 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:01:39.317536 15329 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:39.318336 15329 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbdd7ddae2494eff8bd05f069aae8f27" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40015 } }
I20260808 07:01:39.318447 15329 leader_election.cc:302] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fbdd7ddae2494eff8bd05f069aae8f27; no voters:
I20260808 07:01:39.318766 15329 leader_election.cc:288] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:01:39.318805 15334 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:39.318945 15334 raft_consensus.cc:686] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27 [term 1 LEADER]: Becoming Leader. State: Replica: fbdd7ddae2494eff8bd05f069aae8f27, State: Running, Role: LEADER
I20260808 07:01:39.319039 15334 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbdd7ddae2494eff8bd05f069aae8f27" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40015 } }
I20260808 07:01:39.319095 15329 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:01:39.319411 15334 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbdd7ddae2494eff8bd05f069aae8f27. Latest consensus state: current_term: 1 leader_uuid: "fbdd7ddae2494eff8bd05f069aae8f27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbdd7ddae2494eff8bd05f069aae8f27" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40015 } } }
I20260808 07:01:39.319489 15334 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27 [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:39.319450 15335 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fbdd7ddae2494eff8bd05f069aae8f27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbdd7ddae2494eff8bd05f069aae8f27" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40015 } } }
I20260808 07:01:39.319701 15335 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27 [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:39.319741 15339 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:01:39.320423 15339 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:01:39.321285 15332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.308388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44398 (local address 127.13.207.62:40015)
0808 07:01:39.308603 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:39.308612 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:39.308640 (+ 28us) server_negotiation.cc:407] Connection header received
0808 07:01:39.309128 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:39.309135 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:39.309187 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:39.309318 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:39.310333 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.311454 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.312681 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.313206 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.313336 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.316593 (+ 3257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:39.316598 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:39.316598 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:39.316612 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:39.318877 (+ 2265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:39.319382 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:39.319386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:39.319386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:39.319499 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:39.319737 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:39.319738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:39.319739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:39.320114 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:39.320266 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:39.320775 (+ 509us) server_negotiation.cc:299] Negotiation successful
0808 07:01:39.321056 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":119,"threads_started":1}
I20260808 07:01:39.324081 15339 catalog_manager.cc:1347] Generated new cluster ID: 5152ec0ccdb140669ade9d49faf1b3ff
I20260808 07:01:39.324186 15339 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:01:39.336192 15339 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:01:39.336858 15339 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:01:39.348129 15339 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27: Generated new TSK 0
I20260808 07:01:39.348414 15339 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
I20260808 07:01:39.382426 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40015
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:39959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:39.473843 15355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:39.474157 15355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:39.474205 15355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:39.475666 15355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:39.475749 15355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:39.475788 15355 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:01:39.475842 15355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:01:39.477483 15355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:39959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.207.62:40015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:39.477793 15355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:39.478070 15355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:39.483712 15360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:39.483798 15363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:39.483814 15361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:39.484261 15355 server_base.cc:1034] running on GCE node
I20260808 07:01:39.484480 15355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:39.484766 15355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:39.485978 15355 hybrid_clock.cc:648] HybridClock initialized: now 1786172499485911 us; error 31 us; skew 500 ppm
Aug 08 07:01:39 dist-test-slave-8tdl krb5kdc[15236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172499, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:39.488910 15355 init.cc:377] Logged in from keytab as kudu/127.13.207.1@KRBTEST.COM (short username kudu)
I20260808 07:01:39.489413 15355 webserver.cc:460] Webserver started at http://127.13.207.1:42237/ using document root <none> and password file <none>
I20260808 07:01:39.489686 15355 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:39.489737 15355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:39.489885 15355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:39.491122 15355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "f58811b83a68475599c2a11167f14492"
format_stamp: "Formatted at 2026-08-08 07:01:39 on dist-test-slave-8tdl"
server_key: "4bc5d444066f958f6d814774d52d9e51"
server_key_iv: "c18eb447cc2090009eb9b699ff29f0ef"
server_key_version: "encryptionkey@0"
I20260808 07:01:39.491500 15355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "f58811b83a68475599c2a11167f14492"
format_stamp: "Formatted at 2026-08-08 07:01:39 on dist-test-slave-8tdl"
server_key: "4bc5d444066f958f6d814774d52d9e51"
server_key_iv: "c18eb447cc2090009eb9b699ff29f0ef"
server_key_version: "encryptionkey@0"
I20260808 07:01:39.492992 15355 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:39.493925 15370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:39.494308 15355 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:39.494451 15355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "f58811b83a68475599c2a11167f14492"
format_stamp: "Formatted at 2026-08-08 07:01:39 on dist-test-slave-8tdl"
server_key: "4bc5d444066f958f6d814774d52d9e51"
server_key_iv: "c18eb447cc2090009eb9b699ff29f0ef"
server_key_version: "encryptionkey@0"
I20260808 07:01:39.494554 15355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:39.510898 15355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:39.513495 15355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:39.513686 15355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:39.514278 15355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:39.514340 15355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:39.514375 15355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:39.514407 15355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:39.522490 15355 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:32835
Aug 08 07:01:39 dist-test-slave-8tdl krb5kdc[15236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172499, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:39.522574 15489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:32835 every 8 connection(s)
I20260808 07:01:39.522992 15355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:01:39.527230 15332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.515073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:51473 (local address 127.13.207.62:40015)
0808 07:01:39.515162 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:39.515164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:39.515354 (+ 190us) server_negotiation.cc:407] Connection header received
0808 07:01:39.515937 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:39.515939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:39.515959 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:39.516022 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:39.516880 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.517832 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.519534 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.519947 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.520033 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.523291 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:39.523293 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:39.523293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:39.523304 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:39.525207 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:39.525772 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:39.525773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:39.525773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:39.525838 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:39.526299 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:39.526299 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:39.526300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:39.526441 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:39.526504 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:39.527043 (+ 539us) server_negotiation.cc:299] Negotiation successful
0808 07:01:39.527140 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 07:01:39.527374 15379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.514986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:40015 (local address 127.13.207.1:51473)
0808 07:01:39.515204 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:39.515249 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:39.515819 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:39.516097 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:39.516101 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:39.516495 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:39.516783 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:39.516789 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.517979 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.517980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:39.519407 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:39.519411 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.520066 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.520068 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:39.520250 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.520964 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:39.520975 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:39.523125 (+ 2150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:39.525321 (+ 2196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:39.525328 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:39.525331 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:39.525668 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:39.526104 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:39.526105 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:39.526105 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:39.526223 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:39.526511 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:39.526518 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:39.526651 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 07:01:39.527011 (+ 360us) client_negotiation.cc:240] Negotiation successful
0808 07:01:39.527159 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":103,"threads_started":1}
I20260808 07:01:39.527949 15490 heartbeater.cc:347] Connected to a master server at 127.13.207.62:40015
I20260808 07:01:39.528052 15490 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:39.528266 15490 heartbeater.cc:511] Master 127.13.207.62:40015 requested a full tablet report, sending...
I20260808 07:01:39.528970 15281 ts_manager.cc:194] Registered new tserver with Master: f58811b83a68475599c2a11167f14492 (127.13.207.1:32835)
I20260808 07:01:39.529996 15281 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:51473
I20260808 07:01:39.530252 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 15355
I20260808 07:01:39.530359 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:01:39.530592 14140 external_mini_cluster.cc:1428] Setting key 61effe6e2c45bfa547ab6d5eff07b47b
I20260808 07:01:39.535706 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.529283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:40015 (local address 127.13.207.1:60723)
0808 07:01:39.529422 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:39.529433 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:39.529489 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:39.529637 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:39.529638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:39.529852 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:39.529923 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:39.529925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.531071 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.531072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:39.531606 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:39.531608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.531988 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.531989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:39.532077 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.532556 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:39.532561 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:39.532876 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:39.534525 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:39.534526 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:39.534526 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:39.534768 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:39.535022 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:39.535023 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:39.535023 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:39.535088 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:39.535349 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:39.535351 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:39.535399 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 07:01:39.535485 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 07:01:39.535601 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":74,"threads_started":1}
I20260808 07:01:39.535702 15332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.529393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:60723 (local address 127.13.207.62:40015)
0808 07:01:39.529434 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:39.529437 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:39.529475 (+ 38us) server_negotiation.cc:407] Connection header received
0808 07:01:39.529544 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:39.529545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:39.529557 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:39.529612 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:39.530103 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.530886 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.531704 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.531909 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.531991 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.532992 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:39.532994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:39.532994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:39.533001 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:39.534417 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:39.534878 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:39.534879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:39.534879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:39.534948 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:39.535157 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:39.535158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:39.535158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:39.535275 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:39.535347 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:39.535510 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 07:01:39.535599 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.13.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
I20260808 07:01:39.563647 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40015
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:39959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:39.653388 15499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:39.653635 15499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:39.653662 15499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:39.654940 15499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:39.655001 15499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:39.655026 15499 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:01:39.655046 15499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:01:39.656713 15499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:39959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.207.62:40015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:39.656945 15499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:39.657153 15499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:39.663756 15508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:39.663810 15505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:39.663792 15499 server_base.cc:1034] running on GCE node
W20260808 07:01:39.664170 15506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:39.664562 15499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:39.664865 15499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:39.666052 15499 hybrid_clock.cc:648] HybridClock initialized: now 1786172499666025 us; error 42 us; skew 500 ppm
Aug 08 07:01:39 dist-test-slave-8tdl krb5kdc[15236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172499, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:39.669329 15499 init.cc:377] Logged in from keytab as kudu/127.13.207.2@KRBTEST.COM (short username kudu)
I20260808 07:01:39.669983 15499 webserver.cc:460] Webserver started at http://127.13.207.2:43425/ using document root <none> and password file <none>
I20260808 07:01:39.670264 15499 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:39.670358 15499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:39.670532 15499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:39.671859 15499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "e1fa47d74bcb467584b6eb9044885f01"
format_stamp: "Formatted at 2026-08-08 07:01:39 on dist-test-slave-8tdl"
server_key: "db68ea3e047eebe3537d60083e3ec027"
server_key_iv: "a0139f444136a45b761e40da9c0aeadd"
server_key_version: "encryptionkey@0"
I20260808 07:01:39.672284 15499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "e1fa47d74bcb467584b6eb9044885f01"
format_stamp: "Formatted at 2026-08-08 07:01:39 on dist-test-slave-8tdl"
server_key: "db68ea3e047eebe3537d60083e3ec027"
server_key_iv: "a0139f444136a45b761e40da9c0aeadd"
server_key_version: "encryptionkey@0"
I20260808 07:01:39.673882 15499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:01:39.674927 15515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:39.675249 15499 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:01:39.675384 15499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "e1fa47d74bcb467584b6eb9044885f01"
format_stamp: "Formatted at 2026-08-08 07:01:39 on dist-test-slave-8tdl"
server_key: "db68ea3e047eebe3537d60083e3ec027"
server_key_iv: "a0139f444136a45b761e40da9c0aeadd"
server_key_version: "encryptionkey@0"
I20260808 07:01:39.675469 15499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:39.688853 15499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:39.691913 15499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:39.692126 15499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:39.692706 15499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:39.692773 15499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:39.692837 15499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:39.692868 15499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:01:39 dist-test-slave-8tdl krb5kdc[15236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172499, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:39.700690 15499 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:39515
I20260808 07:01:39.700814 15632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:39515 every 8 connection(s)
I20260808 07:01:39.701340 15499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:01:39.704557 15332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.693345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:32863 (local address 127.13.207.62:40015)
0808 07:01:39.693436 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:39.693439 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:39.693761 (+ 322us) server_negotiation.cc:407] Connection header received
0808 07:01:39.694530 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:39.694532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:39.694547 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:39.694637 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:39.695748 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.696541 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.697743 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.698049 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.698146 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.700993 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:39.700996 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:39.700996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:39.701006 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:39.702869 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:39.703410 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:39.703411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:39.703412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:39.703487 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:39.703785 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:39.703786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:39.703786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:39.703933 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:39.704015 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:39.704339 (+ 324us) server_negotiation.cc:299] Negotiation successful
0808 07:01:39.704444 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 07:01:39.704800 15524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.693310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:40015 (local address 127.13.207.2:32863)
0808 07:01:39.693626 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:39.693659 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:39.694393 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:39.694778 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:39.694783 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:39.695252 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:39.695636 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:39.695645 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.696662 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.696663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:39.697623 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:39.697625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.698172 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.698173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:39.698346 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.699070 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:39.699079 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:39.700832 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:39.703026 (+ 2194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:39.703034 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:39.703036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:39.703313 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:39.703565 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:39.703566 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:39.703566 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:39.703699 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:39.704018 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:39.704025 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:39.704185 (+ 160us) client_negotiation.cc:769] Sending connection context
0808 07:01:39.704316 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 07:01:39.704524 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":124,"threads_started":1}
I20260808 07:01:39.705375 15633 heartbeater.cc:347] Connected to a master server at 127.13.207.62:40015
I20260808 07:01:39.705502 15633 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:39.705706 15633 heartbeater.cc:511] Master 127.13.207.62:40015 requested a full tablet report, sending...
I20260808 07:01:39.706146 15282 ts_manager.cc:194] Registered new tserver with Master: e1fa47d74bcb467584b6eb9044885f01 (127.13.207.2:39515)
I20260808 07:01:39.706555 15282 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:32863
I20260808 07:01:39.710938 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 15499
I20260808 07:01:39.711066 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:01:39.711272 14140 external_mini_cluster.cc:1428] Setting key f142c0142e54c1c979574a221414ea0d
I20260808 07:01:39.712929 15332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.706838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:60181 (local address 127.13.207.62:40015)
0808 07:01:39.706880 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:39.706882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:39.707019 (+ 137us) server_negotiation.cc:407] Connection header received
0808 07:01:39.707050 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:39.707051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:39.707063 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:39.707154 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:39.707507 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.707981 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.708688 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.708869 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.708923 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.709851 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:39.709853 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:39.709853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:39.709859 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:39.711587 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:39.712131 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:39.712132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:39.712132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:39.712188 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:39.712416 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:39.712417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:39.712417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:39.712560 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:39.712611 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:39.712796 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 07:01:39.712863 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:01:39.713079 15640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.706799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:40015 (local address 127.13.207.2:60181)
0808 07:01:39.706933 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:39.706944 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:39.707012 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:39.707169 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:39.707170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:39.707359 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:39.707430 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:39.707432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.708094 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.708095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:39.708610 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:39.708612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.708943 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.708944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:39.709015 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.709434 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:39.709439 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:39.709770 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:39.711676 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:39.711678 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:39.711679 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:39.712044 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:39.712258 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:39.712259 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:39.712259 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:39.712335 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:39.712625 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:39.712626 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:39.712689 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 07:01:39.712773 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 07:01:39.712914 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":77,"threads_started":1}
WARNING: no policy specified for kudu/127.13.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
I20260808 07:01:39.743132 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40015
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:39959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:39.834513 15643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:39.834898 15643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:39.834951 15643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:39.836316 15643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:39.836400 15643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:39.836438 15643 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:01:39.836484 15643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:01:39.838142 15643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:39959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.207.62:40015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:39.838447 15643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:39.838786 15643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:39.844666 15650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:39.844667 15649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:39.844789 15652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:39.845130 15643 server_base.cc:1034] running on GCE node
I20260808 07:01:39.845357 15643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:39.845619 15643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:39.846814 15643 hybrid_clock.cc:648] HybridClock initialized: now 1786172499846801 us; error 44 us; skew 500 ppm
Aug 08 07:01:39 dist-test-slave-8tdl krb5kdc[15236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172499, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:39.849885 15643 init.cc:377] Logged in from keytab as kudu/127.13.207.3@KRBTEST.COM (short username kudu)
I20260808 07:01:39.850517 15643 webserver.cc:460] Webserver started at http://127.13.207.3:40115/ using document root <none> and password file <none>
I20260808 07:01:39.850849 15643 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:39.850943 15643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:39.851131 15643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:39.852392 15643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "a198d9909afc4a1ca15b39b362477d26"
format_stamp: "Formatted at 2026-08-08 07:01:39 on dist-test-slave-8tdl"
server_key: "460c7b955021a734b57989726873bb97"
server_key_iv: "d4f8cd87232ab7990e5993c3af9f8a35"
server_key_version: "encryptionkey@0"
I20260808 07:01:39.852813 15643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "a198d9909afc4a1ca15b39b362477d26"
format_stamp: "Formatted at 2026-08-08 07:01:39 on dist-test-slave-8tdl"
server_key: "460c7b955021a734b57989726873bb97"
server_key_iv: "d4f8cd87232ab7990e5993c3af9f8a35"
server_key_version: "encryptionkey@0"
I20260808 07:01:39.854431 15643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:01:39.855581 15659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:39.855818 15643 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:39.855935 15643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "a198d9909afc4a1ca15b39b362477d26"
format_stamp: "Formatted at 2026-08-08 07:01:39 on dist-test-slave-8tdl"
server_key: "460c7b955021a734b57989726873bb97"
server_key_iv: "d4f8cd87232ab7990e5993c3af9f8a35"
server_key_version: "encryptionkey@0"
I20260808 07:01:39.856032 15643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:39.869573 15643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:39.872354 15643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:39.872586 15643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:39.873164 15643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:39.873232 15643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:39.873282 15643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:39.873308 15643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:01:39 dist-test-slave-8tdl krb5kdc[15236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172499, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:39.881796 15643 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:33337
I20260808 07:01:39.881870 15778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:33337 every 8 connection(s)
I20260808 07:01:39.882268 15643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:01:39.885468 15332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.873744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:49869 (local address 127.13.207.62:40015)
0808 07:01:39.873790 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:39.873793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:39.874039 (+ 246us) server_negotiation.cc:407] Connection header received
0808 07:01:39.874608 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:39.874610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:39.874624 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:39.874727 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:39.875767 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.876639 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.878545 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.878853 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.878929 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.882200 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:39.882202 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:39.882202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:39.882212 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:39.883950 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:39.884426 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:39.884428 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:39.884428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:39.884493 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:39.884755 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:39.884757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:39.884757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:39.884902 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:39.884970 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:39.885311 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 07:01:39.885393 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:01:39.885759 15668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.873738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:40015 (local address 127.13.207.3:49869)
0808 07:01:39.873903 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:39.873940 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:39.874487 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:39.874858 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:39.874863 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:39.875296 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:39.875646 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:39.875653 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.876805 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.876807 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:39.878423 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:39.878427 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.878953 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.878954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:39.879181 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.880001 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:39.880011 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:39.882054 (+ 2043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:39.884074 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:39.884081 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:39.884083 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:39.884339 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:39.884563 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:39.884563 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:39.884564 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:39.884669 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:39.885008 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:39.885015 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:39.885161 (+ 146us) client_negotiation.cc:769] Sending connection context
0808 07:01:39.885269 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 07:01:39.885415 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":101,"threads_started":1}
I20260808 07:01:39.886466 15779 heartbeater.cc:347] Connected to a master server at 127.13.207.62:40015
I20260808 07:01:39.886562 15779 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:39.886813 15779 heartbeater.cc:511] Master 127.13.207.62:40015 requested a full tablet report, sending...
I20260808 07:01:39.887405 15282 ts_manager.cc:194] Registered new tserver with Master: a198d9909afc4a1ca15b39b362477d26 (127.13.207.3:33337)
I20260808 07:01:39.888120 15282 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:49869
I20260808 07:01:39.890007 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 15643
I20260808 07:01:39.890111 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:01:39.890309 14140 external_mini_cluster.cc:1428] Setting key 6c2651bf7a0b8d1e9f53a358425991bd
I20260808 07:01:39.891449 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:01:39.895023 15332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.887899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:60109 (local address 127.13.207.62:40015)
0808 07:01:39.887964 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:39.887966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:39.888114 (+ 148us) server_negotiation.cc:407] Connection header received
0808 07:01:39.888159 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:39.888160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:39.888173 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:39.888230 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:39.888683 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.889226 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.890183 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.890516 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.890563 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.891906 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:39.891907 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:39.891907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:39.891916 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:39.893523 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:39.894086 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:39.894088 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:39.894088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:39.894156 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:39.894403 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:39.894404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:39.894404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:39.894534 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:39.894593 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:39.894866 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 07:01:39.894948 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:01:39.895109 15785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.887818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:40015 (local address 127.13.207.3:60109)
0808 07:01:39.887966 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:39.887977 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:39.888104 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:39.888288 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:39.888289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:39.888525 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:39.888592 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:39.888594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.889343 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.889343 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:39.890056 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:39.890060 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.890688 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.890713 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:39.890811 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.891359 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:39.891365 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:39.891810 (+ 445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:39.893685 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:39.893687 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:39.893687 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:39.893980 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:39.894241 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:39.894242 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:39.894243 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:39.894328 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:39.894607 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:39.894607 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:39.894727 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 07:01:39.894834 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 07:01:39.894979 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
Aug 08 07:01:39 dist-test-slave-8tdl krb5kdc[15236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172499, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:01:39 dist-test-slave-8tdl krb5kdc[15236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172499, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:39.921092 15332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.912082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44408 (local address 127.13.207.62:40015)
0808 07:01:39.912145 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:39.912147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:39.912157 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:01:39.912223 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:39.912224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:39.912234 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:39.912287 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:39.913017 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.913530 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.914420 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.914631 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.914766 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.917580 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:39.917582 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:39.917583 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:39.917601 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:39.919655 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:39.920197 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:39.920199 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:39.920199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:39.920279 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:39.920494 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:39.920495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:39.920495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:39.920633 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:39.920723 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:39.920909 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 07:01:39.920992 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 07:01:39.923153 15282 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44408:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:01:39.924916 15282 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:01:39.932865 15796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.928066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:32835 (local address 127.0.0.1:56592)
0808 07:01:39.928383 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:39.928457 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:39.928538 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:39.929058 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:39.929058 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:39.929061 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:39.929101 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:39.929103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.930338 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.930340 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:39.931734 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:39.931737 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.932485 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.932486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:39.932583 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.932605 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 07:01:39.932687 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:01:39.932751 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":58,"threads_started":1}
I20260808 07:01:39.932868 15800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.928349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56592 (local address 127.13.207.1:32835)
0808 07:01:39.928726 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:39.928732 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:39.928743 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:01:39.928787 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:39.928790 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:39.928806 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:39.928895 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:39.929210 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.930118 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.931825 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.932379 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.932482 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.932557 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:39.932718 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 07:01:39.932760 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":95,"threads_started":1}
I20260808 07:01:39.933944 15797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.928068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:39515 (local address 127.0.0.1:51540)
0808 07:01:39.928415 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:39.928459 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:39.928587 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:39.928987 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:39.928991 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:39.928993 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:39.929088 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:39.929091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.930888 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.930889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:39.932524 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:39.932528 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.933675 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.933676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:39.933747 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.933752 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:01:39.933805 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 07:01:39.933852 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":5,"thread_start_us":123,"threads_started":1}
I20260808 07:01:39.933917 15422 tablet_service.cc:1467] Processing CreateTablet for tablet a80263962eb348b7be66e20e2ce2e348 (DEFAULT_TABLE table=test-table [id=ee24ba1d7bbe4528bef0c80ff4d6bf36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:39.934345 15799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.928197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51540 (local address 127.13.207.2:39515)
0808 07:01:39.928763 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:39.928767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:39.928776 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:01:39.928811 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:39.928813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:39.928827 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:39.928896 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:39.929192 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.930664 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.932648 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.933505 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.933580 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.933636 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:39.934167 (+ 531us) server_negotiation.cc:299] Negotiation successful
0808 07:01:39.934217 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":112,"threads_started":1}
I20260808 07:01:39.934355 15422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a80263962eb348b7be66e20e2ce2e348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:39.934481 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.928162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:33337 (local address 127.0.0.1:40820)
0808 07:01:39.928543 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:39.928553 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:39.928602 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:39.928998 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:39.928999 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:39.929002 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:39.929091 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:39.929094 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.931742 (+ 2648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.931744 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:39.933036 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:39.933039 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.934204 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.934205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:39.934275 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.934281 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:01:39.934351 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 07:01:39.934403 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":121,"threads_started":1}
I20260808 07:01:39.934753 15801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.928450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40820 (local address 127.13.207.3:33337)
0808 07:01:39.928719 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:39.928725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:39.928739 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:01:39.928784 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:39.928787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:39.928804 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:39.928889 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:39.930385 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.931600 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.933152 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.934020 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.934101 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.934163 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:39.934385 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 07:01:39.934622 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":109,"threads_started":1}
I20260808 07:01:39.935374 15565 tablet_service.cc:1467] Processing CreateTablet for tablet a80263962eb348b7be66e20e2ce2e348 (DEFAULT_TABLE table=test-table [id=ee24ba1d7bbe4528bef0c80ff4d6bf36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:39.935448 15709 tablet_service.cc:1467] Processing CreateTablet for tablet a80263962eb348b7be66e20e2ce2e348 (DEFAULT_TABLE table=test-table [id=ee24ba1d7bbe4528bef0c80ff4d6bf36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:39.935748 15565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a80263962eb348b7be66e20e2ce2e348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:39.935827 15709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a80263962eb348b7be66e20e2ce2e348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:39.938864 15802 tablet_bootstrap.cc:492] T a80263962eb348b7be66e20e2ce2e348 P a198d9909afc4a1ca15b39b362477d26: Bootstrap starting.
I20260808 07:01:39.939337 15803 tablet_bootstrap.cc:492] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492: Bootstrap starting.
I20260808 07:01:39.939790 15804 tablet_bootstrap.cc:492] T a80263962eb348b7be66e20e2ce2e348 P e1fa47d74bcb467584b6eb9044885f01: Bootstrap starting.
I20260808 07:01:39.940202 15803 tablet_bootstrap.cc:654] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:39.940310 15802 tablet_bootstrap.cc:654] T a80263962eb348b7be66e20e2ce2e348 P a198d9909afc4a1ca15b39b362477d26: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:39.940570 15803 log.cc:824] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:39.940631 15804 tablet_bootstrap.cc:654] T a80263962eb348b7be66e20e2ce2e348 P e1fa47d74bcb467584b6eb9044885f01: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:39.940706 15802 log.cc:824] T a80263962eb348b7be66e20e2ce2e348 P a198d9909afc4a1ca15b39b362477d26: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:39.941027 15804 log.cc:824] T a80263962eb348b7be66e20e2ce2e348 P e1fa47d74bcb467584b6eb9044885f01: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:39.941515 15803 tablet_bootstrap.cc:492] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492: No bootstrap required, opened a new log
I20260808 07:01:39.941604 15803 ts_tablet_manager.cc:1397] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:01:39.941877 15804 tablet_bootstrap.cc:492] T a80263962eb348b7be66e20e2ce2e348 P e1fa47d74bcb467584b6eb9044885f01: No bootstrap required, opened a new log
I20260808 07:01:39.941957 15804 ts_tablet_manager.cc:1397] T a80263962eb348b7be66e20e2ce2e348 P e1fa47d74bcb467584b6eb9044885f01: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:01:39.943078 15802 tablet_bootstrap.cc:492] T a80263962eb348b7be66e20e2ce2e348 P a198d9909afc4a1ca15b39b362477d26: No bootstrap required, opened a new log
I20260808 07:01:39.943182 15802 ts_tablet_manager.cc:1397] T a80263962eb348b7be66e20e2ce2e348 P a198d9909afc4a1ca15b39b362477d26: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260808 07:01:39.944065 15803 raft_consensus.cc:348] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:39.944274 15803 raft_consensus.cc:374] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:39.944195 15804 raft_consensus.cc:348] T a80263962eb348b7be66e20e2ce2e348 P e1fa47d74bcb467584b6eb9044885f01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:39.944316 15803 raft_consensus.cc:729] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f58811b83a68475599c2a11167f14492, State: Initialized, Role: FOLLOWER
I20260808 07:01:39.944336 15804 raft_consensus.cc:374] T a80263962eb348b7be66e20e2ce2e348 P e1fa47d74bcb467584b6eb9044885f01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:39.944371 15804 raft_consensus.cc:729] T a80263962eb348b7be66e20e2ce2e348 P e1fa47d74bcb467584b6eb9044885f01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1fa47d74bcb467584b6eb9044885f01, State: Initialized, Role: FOLLOWER
I20260808 07:01:39.944468 15803 consensus_queue.cc:260] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:39.944550 15804 consensus_queue.cc:260] T a80263962eb348b7be66e20e2ce2e348 P e1fa47d74bcb467584b6eb9044885f01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:39.944813 15804 ts_tablet_manager.cc:1428] T a80263962eb348b7be66e20e2ce2e348 P e1fa47d74bcb467584b6eb9044885f01: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:01:39.944813 15803 ts_tablet_manager.cc:1428] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:01:39.944959 15633 heartbeater.cc:503] Master 127.13.207.62:40015 was elected leader, sending a full tablet report...
I20260808 07:01:39.944969 15490 heartbeater.cc:503] Master 127.13.207.62:40015 was elected leader, sending a full tablet report...
I20260808 07:01:39.945176 15802 raft_consensus.cc:348] T a80263962eb348b7be66e20e2ce2e348 P a198d9909afc4a1ca15b39b362477d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:39.945371 15802 raft_consensus.cc:374] T a80263962eb348b7be66e20e2ce2e348 P a198d9909afc4a1ca15b39b362477d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:39.945420 15802 raft_consensus.cc:729] T a80263962eb348b7be66e20e2ce2e348 P a198d9909afc4a1ca15b39b362477d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a198d9909afc4a1ca15b39b362477d26, State: Initialized, Role: FOLLOWER
I20260808 07:01:39.945539 15802 consensus_queue.cc:260] T a80263962eb348b7be66e20e2ce2e348 P a198d9909afc4a1ca15b39b362477d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:39.945873 15802 ts_tablet_manager.cc:1428] T a80263962eb348b7be66e20e2ce2e348 P a198d9909afc4a1ca15b39b362477d26: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:01:39.945926 15779 heartbeater.cc:503] Master 127.13.207.62:40015 was elected leader, sending a full tablet report...
W20260808 07:01:39.952169 15634 tablet.cc:2369] T a80263962eb348b7be66e20e2ce2e348 P e1fa47d74bcb467584b6eb9044885f01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:01:39.987162 15808 raft_consensus.cc:482] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:01:39.987345 15808 raft_consensus.cc:504] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:39.987972 15808 leader_election.cc:288] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1fa47d74bcb467584b6eb9044885f01 (127.13.207.2:39515), a198d9909afc4a1ca15b39b362477d26 (127.13.207.3:33337)
I20260808 07:01:39.991163 15801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.988253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:49063 (local address 127.13.207.3:33337)
0808 07:01:39.988302 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:39.988305 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:39.988465 (+ 160us) server_negotiation.cc:407] Connection header received
0808 07:01:39.988555 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:39.988556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:39.988566 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:39.988614 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:39.988785 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.989309 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.990334 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.990774 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.990857 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.990867 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:39.991037 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 07:01:39.991078 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:01:39.991189 15811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.988166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:33337 (local address 127.13.207.1:49063)
0808 07:01:39.988379 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:39.988389 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:39.988487 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:39.988626 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:39.988627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:39.988629 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:39.988710 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:39.988712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.989407 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.989408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:39.990227 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:39.990229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.990857 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.990858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:39.990942 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.990951 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 07:01:39.991018 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 07:01:39.991062 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":20,"thread_start_us":114,"threads_started":1}
I20260808 07:01:39.991194 15799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.988317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:38081 (local address 127.13.207.2:39515)
0808 07:01:39.988366 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:39.988368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:39.988377 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:01:39.988406 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:39.988407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:39.988417 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:39.988516 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:39.988705 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.989240 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.990355 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.990782 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.990880 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.990889 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:39.991075 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 07:01:39.991116 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:01:39.991281 15379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:39.988166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:39515 (local address 127.13.207.1:38081)
0808 07:01:39.988252 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:39.988267 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:39.988329 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:39.988529 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:39.988530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:39.988532 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:39.988617 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:39.988619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.989345 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.989346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:39.990246 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:39.990249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:39.990880 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:39.990881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:39.990990 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:39.990993 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:01:39.991069 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 07:01:39.991113 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":31}
I20260808 07:01:39.991839 15586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a80263962eb348b7be66e20e2ce2e348" candidate_uuid: "f58811b83a68475599c2a11167f14492" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1fa47d74bcb467584b6eb9044885f01" is_pre_election: true
I20260808 07:01:39.991839 15730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a80263962eb348b7be66e20e2ce2e348" candidate_uuid: "f58811b83a68475599c2a11167f14492" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a198d9909afc4a1ca15b39b362477d26" is_pre_election: true
I20260808 07:01:39.992033 15730 raft_consensus.cc:2454] T a80263962eb348b7be66e20e2ce2e348 P a198d9909afc4a1ca15b39b362477d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f58811b83a68475599c2a11167f14492 in term 0.
I20260808 07:01:39.992033 15586 raft_consensus.cc:2454] T a80263962eb348b7be66e20e2ce2e348 P e1fa47d74bcb467584b6eb9044885f01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f58811b83a68475599c2a11167f14492 in term 0.
I20260808 07:01:39.992307 15373 leader_election.cc:302] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a198d9909afc4a1ca15b39b362477d26, f58811b83a68475599c2a11167f14492; no voters:
I20260808 07:01:39.992522 15808 raft_consensus.cc:2781] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:01:39.992596 15808 raft_consensus.cc:482] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:01:39.992635 15808 raft_consensus.cc:3037] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:39.993642 15808 raft_consensus.cc:504] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:39.993820 15808 leader_election.cc:288] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492 [CANDIDATE]: Term 1 election: Requested vote from peers e1fa47d74bcb467584b6eb9044885f01 (127.13.207.2:39515), a198d9909afc4a1ca15b39b362477d26 (127.13.207.3:33337)
I20260808 07:01:39.994099 15730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a80263962eb348b7be66e20e2ce2e348" candidate_uuid: "f58811b83a68475599c2a11167f14492" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a198d9909afc4a1ca15b39b362477d26"
I20260808 07:01:39.994153 15586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a80263962eb348b7be66e20e2ce2e348" candidate_uuid: "f58811b83a68475599c2a11167f14492" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1fa47d74bcb467584b6eb9044885f01"
I20260808 07:01:39.994192 15730 raft_consensus.cc:3037] T a80263962eb348b7be66e20e2ce2e348 P a198d9909afc4a1ca15b39b362477d26 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:39.994269 15586 raft_consensus.cc:3037] T a80263962eb348b7be66e20e2ce2e348 P e1fa47d74bcb467584b6eb9044885f01 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:39.995275 15730 raft_consensus.cc:2454] T a80263962eb348b7be66e20e2ce2e348 P a198d9909afc4a1ca15b39b362477d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f58811b83a68475599c2a11167f14492 in term 1.
I20260808 07:01:39.995453 15586 raft_consensus.cc:2454] T a80263962eb348b7be66e20e2ce2e348 P e1fa47d74bcb467584b6eb9044885f01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f58811b83a68475599c2a11167f14492 in term 1.
I20260808 07:01:39.995473 15373 leader_election.cc:302] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a198d9909afc4a1ca15b39b362477d26, f58811b83a68475599c2a11167f14492; no voters:
I20260808 07:01:39.995594 15808 raft_consensus.cc:2781] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:39.995738 15808 raft_consensus.cc:686] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492 [term 1 LEADER]: Becoming Leader. State: Replica: f58811b83a68475599c2a11167f14492, State: Running, Role: LEADER
I20260808 07:01:39.995884 15808 consensus_queue.cc:237] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:39.996919 15280 catalog_manager.cc:5495] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492 reported cstate change: term changed from 0 to 1, leader changed from <none> to f58811b83a68475599c2a11167f14492 (127.13.207.1). New cstate: current_term: 1 leader_uuid: "f58811b83a68475599c2a11167f14492" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } health_report { overall_health: UNKNOWN } } }
I20260808 07:01:40.011272 15800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:40.007912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56608 (local address 127.13.207.1:32835)
0808 07:01:40.008028 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:40.008030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:40.008043 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:01:40.008087 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:40.008088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:40.008108 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:40.008198 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:01:40.008517 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.009066 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.010011 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.010313 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.010388 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:40.010820 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:01:40.010949 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:01:40.011021 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:40.011109 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 07:01:40.011132 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 07:01:40.013772 15586 raft_consensus.cc:1261] T a80263962eb348b7be66e20e2ce2e348 P e1fa47d74bcb467584b6eb9044885f01 [term 1 FOLLOWER]: Refusing update from remote peer f58811b83a68475599c2a11167f14492: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:01:40.013797 15730 raft_consensus.cc:1261] T a80263962eb348b7be66e20e2ce2e348 P a198d9909afc4a1ca15b39b362477d26 [term 1 FOLLOWER]: Refusing update from remote peer f58811b83a68475599c2a11167f14492: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:01:40.014207 15812 consensus_queue.cc:1035] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:01:40.014351 15808 consensus_queue.cc:1035] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:01:40.021951 15280 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:01:40.022068 15280 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:01:40.023161 15280 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27: Sending DeleteTablet for 3 replicas of tablet a80263962eb348b7be66e20e2ce2e348
I20260808 07:01:40.023787 15422 tablet_service.cc:1514] Processing DeleteTablet for tablet a80263962eb348b7be66e20e2ce2e348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:40 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:56592
I20260808 07:01:40.023787 15565 tablet_service.cc:1514] Processing DeleteTablet for tablet a80263962eb348b7be66e20e2ce2e348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:40 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:51540
I20260808 07:01:40.023787 15709 tablet_service.cc:1514] Processing DeleteTablet for tablet a80263962eb348b7be66e20e2ce2e348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:40 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:40820
I20260808 07:01:40.024250 15826 tablet_replica.cc:330] T a80263962eb348b7be66e20e2ce2e348 P a198d9909afc4a1ca15b39b362477d26: stopping tablet replica
I20260808 07:01:40.024405 15825 tablet_replica.cc:330] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492: stopping tablet replica
I20260808 07:01:40.024519 15825 raft_consensus.cc:2229] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:01:40.024528 15280 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44408:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 07:01:40.024634 15825 raft_consensus.cc:2258] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 07:01:40.024710 15280 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:01:40.024803 15827 tablet_replica.cc:330] T a80263962eb348b7be66e20e2ce2e348 P e1fa47d74bcb467584b6eb9044885f01: stopping tablet replica
I20260808 07:01:40.024917 15827 raft_consensus.cc:2229] T a80263962eb348b7be66e20e2ce2e348 P e1fa47d74bcb467584b6eb9044885f01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:01:40.025316 15827 raft_consensus.cc:2258] T a80263962eb348b7be66e20e2ce2e348 P e1fa47d74bcb467584b6eb9044885f01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:01:40.025534 15826 raft_consensus.cc:2229] T a80263962eb348b7be66e20e2ce2e348 P a198d9909afc4a1ca15b39b362477d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:01:40.025673 15826 raft_consensus.cc:2258] T a80263962eb348b7be66e20e2ce2e348 P a198d9909afc4a1ca15b39b362477d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:01:40.026189 15422 tablet_service.cc:1467] Processing CreateTablet for tablet 346171a1a8fd43519d1585336cc70a2c (DEFAULT_TABLE table=test-table [id=6d445185f75a40cd9968a9395cb3686c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:40.026197 15709 tablet_service.cc:1467] Processing CreateTablet for tablet 346171a1a8fd43519d1585336cc70a2c (DEFAULT_TABLE table=test-table [id=6d445185f75a40cd9968a9395cb3686c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:40.026330 15709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 346171a1a8fd43519d1585336cc70a2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:40.026331 15422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 346171a1a8fd43519d1585336cc70a2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:40.026331 15565 tablet_service.cc:1467] Processing CreateTablet for tablet 346171a1a8fd43519d1585336cc70a2c (DEFAULT_TABLE table=test-table [id=6d445185f75a40cd9968a9395cb3686c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:40.026466 15565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 346171a1a8fd43519d1585336cc70a2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:40.026335 15825 ts_tablet_manager.cc:1905] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:01:40.027813 15803 tablet_bootstrap.cc:492] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492: Bootstrap starting.
I20260808 07:01:40.027817 15802 tablet_bootstrap.cc:492] T 346171a1a8fd43519d1585336cc70a2c P a198d9909afc4a1ca15b39b362477d26: Bootstrap starting.
I20260808 07:01:40.028283 15803 tablet_bootstrap.cc:654] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:40.028362 15802 tablet_bootstrap.cc:654] T 346171a1a8fd43519d1585336cc70a2c P a198d9909afc4a1ca15b39b362477d26: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:40.028607 15825 ts_tablet_manager.cc:1918] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:01:40.028684 15804 tablet_bootstrap.cc:492] T 346171a1a8fd43519d1585336cc70a2c P e1fa47d74bcb467584b6eb9044885f01: Bootstrap starting.
I20260808 07:01:40.028728 15825 log.cc:1196] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-0/wal/wals/a80263962eb348b7be66e20e2ce2e348
I20260808 07:01:40.029014 15825 ts_tablet_manager.cc:1939] T a80263962eb348b7be66e20e2ce2e348 P f58811b83a68475599c2a11167f14492: Deleting consensus metadata
I20260808 07:01:40.029116 15804 tablet_bootstrap.cc:654] T 346171a1a8fd43519d1585336cc70a2c P e1fa47d74bcb467584b6eb9044885f01: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:40.029621 15269 catalog_manager.cc:4841] TS f58811b83a68475599c2a11167f14492 (127.13.207.1:32835): tablet a80263962eb348b7be66e20e2ce2e348 (table test-table [id=ee24ba1d7bbe4528bef0c80ff4d6bf36]) successfully deleted
I20260808 07:01:40.029847 15804 tablet_bootstrap.cc:492] T 346171a1a8fd43519d1585336cc70a2c P e1fa47d74bcb467584b6eb9044885f01: No bootstrap required, opened a new log
I20260808 07:01:40.029892 15804 ts_tablet_manager.cc:1397] T 346171a1a8fd43519d1585336cc70a2c P e1fa47d74bcb467584b6eb9044885f01: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:40.030051 15804 raft_consensus.cc:348] T 346171a1a8fd43519d1585336cc70a2c P e1fa47d74bcb467584b6eb9044885f01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:40.030112 15804 raft_consensus.cc:374] T 346171a1a8fd43519d1585336cc70a2c P e1fa47d74bcb467584b6eb9044885f01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:40.030125 15804 raft_consensus.cc:729] T 346171a1a8fd43519d1585336cc70a2c P e1fa47d74bcb467584b6eb9044885f01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1fa47d74bcb467584b6eb9044885f01, State: Initialized, Role: FOLLOWER
I20260808 07:01:40.030191 15804 consensus_queue.cc:260] T 346171a1a8fd43519d1585336cc70a2c P e1fa47d74bcb467584b6eb9044885f01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:40.030284 15804 ts_tablet_manager.cc:1428] T 346171a1a8fd43519d1585336cc70a2c P e1fa47d74bcb467584b6eb9044885f01: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:40.030256 15826 ts_tablet_manager.cc:1905] T a80263962eb348b7be66e20e2ce2e348 P a198d9909afc4a1ca15b39b362477d26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:01:40.032254 15826 ts_tablet_manager.cc:1918] T a80263962eb348b7be66e20e2ce2e348 P a198d9909afc4a1ca15b39b362477d26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:01:40.032356 15826 log.cc:1196] T a80263962eb348b7be66e20e2ce2e348 P a198d9909afc4a1ca15b39b362477d26: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/a80263962eb348b7be66e20e2ce2e348
I20260808 07:01:40.032718 15826 ts_tablet_manager.cc:1939] T a80263962eb348b7be66e20e2ce2e348 P a198d9909afc4a1ca15b39b362477d26: Deleting consensus metadata
I20260808 07:01:40.033238 15827 ts_tablet_manager.cc:1905] T a80263962eb348b7be66e20e2ce2e348 P e1fa47d74bcb467584b6eb9044885f01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:01:40.033641 15268 catalog_manager.cc:4841] TS a198d9909afc4a1ca15b39b362477d26 (127.13.207.3:33337): tablet a80263962eb348b7be66e20e2ce2e348 (table test-table [id=ee24ba1d7bbe4528bef0c80ff4d6bf36]) successfully deleted
I20260808 07:01:40.035010 15827 ts_tablet_manager.cc:1918] T a80263962eb348b7be66e20e2ce2e348 P e1fa47d74bcb467584b6eb9044885f01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:01:40.035104 15827 log.cc:1196] T a80263962eb348b7be66e20e2ce2e348 P e1fa47d74bcb467584b6eb9044885f01: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-1/wal/wals/a80263962eb348b7be66e20e2ce2e348
I20260808 07:01:40.035296 15802 tablet_bootstrap.cc:492] T 346171a1a8fd43519d1585336cc70a2c P a198d9909afc4a1ca15b39b362477d26: No bootstrap required, opened a new log
I20260808 07:01:40.035360 15802 ts_tablet_manager.cc:1397] T 346171a1a8fd43519d1585336cc70a2c P a198d9909afc4a1ca15b39b362477d26: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260808 07:01:40.035403 15827 ts_tablet_manager.cc:1939] T a80263962eb348b7be66e20e2ce2e348 P e1fa47d74bcb467584b6eb9044885f01: Deleting consensus metadata
I20260808 07:01:40.035508 15803 tablet_bootstrap.cc:492] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492: No bootstrap required, opened a new log
I20260808 07:01:40.035553 15803 ts_tablet_manager.cc:1397] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260808 07:01:40.035537 15802 raft_consensus.cc:348] T 346171a1a8fd43519d1585336cc70a2c P a198d9909afc4a1ca15b39b362477d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:40.035615 15802 raft_consensus.cc:374] T 346171a1a8fd43519d1585336cc70a2c P a198d9909afc4a1ca15b39b362477d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:40.035641 15802 raft_consensus.cc:729] T 346171a1a8fd43519d1585336cc70a2c P a198d9909afc4a1ca15b39b362477d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a198d9909afc4a1ca15b39b362477d26, State: Initialized, Role: FOLLOWER
I20260808 07:01:40.035696 15802 consensus_queue.cc:260] T 346171a1a8fd43519d1585336cc70a2c P a198d9909afc4a1ca15b39b362477d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:40.035794 15802 ts_tablet_manager.cc:1428] T 346171a1a8fd43519d1585336cc70a2c P a198d9909afc4a1ca15b39b362477d26: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:40.035748 15803 raft_consensus.cc:348] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:40.035838 15803 raft_consensus.cc:374] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:40.035866 15803 raft_consensus.cc:729] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f58811b83a68475599c2a11167f14492, State: Initialized, Role: FOLLOWER
I20260808 07:01:40.035915 15803 consensus_queue.cc:260] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:40.036263 15803 ts_tablet_manager.cc:1428] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:40.037313 15267 catalog_manager.cc:4841] TS e1fa47d74bcb467584b6eb9044885f01 (127.13.207.2:39515): tablet a80263962eb348b7be66e20e2ce2e348 (table test-table [id=ee24ba1d7bbe4528bef0c80ff4d6bf36]) successfully deleted
I20260808 07:01:40.097730 15822 raft_consensus.cc:482] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:01:40.097863 15822 raft_consensus.cc:504] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:40.098110 15822 leader_election.cc:288] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1fa47d74bcb467584b6eb9044885f01 (127.13.207.2:39515), a198d9909afc4a1ca15b39b362477d26 (127.13.207.3:33337)
I20260808 07:01:40.098331 15586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346171a1a8fd43519d1585336cc70a2c" candidate_uuid: "f58811b83a68475599c2a11167f14492" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1fa47d74bcb467584b6eb9044885f01" is_pre_election: true
I20260808 07:01:40.098564 15586 raft_consensus.cc:2454] T 346171a1a8fd43519d1585336cc70a2c P e1fa47d74bcb467584b6eb9044885f01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f58811b83a68475599c2a11167f14492 in term 0.
I20260808 07:01:40.098542 15730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346171a1a8fd43519d1585336cc70a2c" candidate_uuid: "f58811b83a68475599c2a11167f14492" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a198d9909afc4a1ca15b39b362477d26" is_pre_election: true
I20260808 07:01:40.098656 15730 raft_consensus.cc:2454] T 346171a1a8fd43519d1585336cc70a2c P a198d9909afc4a1ca15b39b362477d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f58811b83a68475599c2a11167f14492 in term 0.
I20260808 07:01:40.098825 15372 leader_election.cc:302] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e1fa47d74bcb467584b6eb9044885f01, f58811b83a68475599c2a11167f14492; no voters:
I20260808 07:01:40.098938 15822 raft_consensus.cc:2781] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:01:40.099236 15822 raft_consensus.cc:482] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:01:40.099263 15822 raft_consensus.cc:3037] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:40.100296 15822 raft_consensus.cc:504] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:40.100594 15822 leader_election.cc:288] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492 [CANDIDATE]: Term 1 election: Requested vote from peers e1fa47d74bcb467584b6eb9044885f01 (127.13.207.2:39515), a198d9909afc4a1ca15b39b362477d26 (127.13.207.3:33337)
I20260808 07:01:40.100690 15586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346171a1a8fd43519d1585336cc70a2c" candidate_uuid: "f58811b83a68475599c2a11167f14492" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1fa47d74bcb467584b6eb9044885f01"
I20260808 07:01:40.100800 15586 raft_consensus.cc:3037] T 346171a1a8fd43519d1585336cc70a2c P e1fa47d74bcb467584b6eb9044885f01 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:40.100826 15730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346171a1a8fd43519d1585336cc70a2c" candidate_uuid: "f58811b83a68475599c2a11167f14492" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a198d9909afc4a1ca15b39b362477d26"
I20260808 07:01:40.100916 15730 raft_consensus.cc:3037] T 346171a1a8fd43519d1585336cc70a2c P a198d9909afc4a1ca15b39b362477d26 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:40.101589 15586 raft_consensus.cc:2454] T 346171a1a8fd43519d1585336cc70a2c P e1fa47d74bcb467584b6eb9044885f01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f58811b83a68475599c2a11167f14492 in term 1.
I20260808 07:01:40.101750 15730 raft_consensus.cc:2454] T 346171a1a8fd43519d1585336cc70a2c P a198d9909afc4a1ca15b39b362477d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f58811b83a68475599c2a11167f14492 in term 1.
I20260808 07:01:40.101922 15372 leader_election.cc:302] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e1fa47d74bcb467584b6eb9044885f01, f58811b83a68475599c2a11167f14492; no voters:
I20260808 07:01:40.102155 15822 raft_consensus.cc:2781] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:40.102240 15822 raft_consensus.cc:686] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492 [term 1 LEADER]: Becoming Leader. State: Replica: f58811b83a68475599c2a11167f14492, State: Running, Role: LEADER
I20260808 07:01:40.102396 15822 consensus_queue.cc:237] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:40.103142 15280 catalog_manager.cc:5495] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492 reported cstate change: term changed from 0 to 1, leader changed from <none> to f58811b83a68475599c2a11167f14492 (127.13.207.1). New cstate: current_term: 1 leader_uuid: "f58811b83a68475599c2a11167f14492" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } health_report { overall_health: UNKNOWN } } }
W20260808 07:01:40.133046 15780 tablet.cc:2369] T 346171a1a8fd43519d1585336cc70a2c P a198d9909afc4a1ca15b39b362477d26: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 07:01:40 dist-test-slave-8tdl krb5kdc[15236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172499, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:40.156569 15332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:40.147704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44412 (local address 127.13.207.62:40015)
0808 07:01:40.147779 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:40.147783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:40.147992 (+ 209us) server_negotiation.cc:407] Connection header received
0808 07:01:40.148044 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:40.148045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:40.148060 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:40.148133 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:40.148711 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.149441 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.150232 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.150538 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.150612 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:40.153417 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:40.153419 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:40.153419 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:40.153427 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:40.155102 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:40.155650 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:40.155651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:40.155652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:40.155721 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:40.155972 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:40.155974 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:40.155974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:40.156122 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:40.156197 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:40.156366 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 07:01:40.156494 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 07:01:40.156688 15837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:40.147591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:40015 (local address 127.0.0.1:44412)
0808 07:01:40.147791 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:40.147805 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:40.147927 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:40.148218 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:40.148219 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:40.148548 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:40.148604 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:40.148605 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.149552 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.149553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:40.150138 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:40.150140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.150644 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.150645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:40.150816 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:40.151494 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:40.151505 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:40.153159 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:40.155193 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:40.155204 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:40.155207 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:40.155483 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:40.155804 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:40.155805 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:40.155805 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:40.155879 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:40.156199 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:40.156208 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:40.156268 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 07:01:40.156346 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:01:40.156593 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":116,"thread_start_us":129,"threads_started":1}
I20260808 07:01:40.159826 15280 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:44412:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 07:01:40.160089 15280 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:01:40.161923 15565 tablet_service.cc:1467] Processing CreateTablet for tablet 10dc7e6a1e8042759810a8de9ddbd390 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cc0e87fc52a44991bd6a0704108b532d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:01:40.161923 15709 tablet_service.cc:1467] Processing CreateTablet for tablet 10dc7e6a1e8042759810a8de9ddbd390 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cc0e87fc52a44991bd6a0704108b532d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:01:40.161935 15422 tablet_service.cc:1467] Processing CreateTablet for tablet 10dc7e6a1e8042759810a8de9ddbd390 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cc0e87fc52a44991bd6a0704108b532d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:01:40.162113 15565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10dc7e6a1e8042759810a8de9ddbd390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:40.162133 15422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10dc7e6a1e8042759810a8de9ddbd390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:40.162155 15709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10dc7e6a1e8042759810a8de9ddbd390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:40.163779 15803 tablet_bootstrap.cc:492] T 10dc7e6a1e8042759810a8de9ddbd390 P f58811b83a68475599c2a11167f14492: Bootstrap starting.
I20260808 07:01:40.163820 15804 tablet_bootstrap.cc:492] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01: Bootstrap starting.
I20260808 07:01:40.163820 15802 tablet_bootstrap.cc:492] T 10dc7e6a1e8042759810a8de9ddbd390 P a198d9909afc4a1ca15b39b362477d26: Bootstrap starting.
I20260808 07:01:40.164127 15803 tablet_bootstrap.cc:654] T 10dc7e6a1e8042759810a8de9ddbd390 P f58811b83a68475599c2a11167f14492: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:40.164281 15804 tablet_bootstrap.cc:654] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:40.164338 15802 tablet_bootstrap.cc:654] T 10dc7e6a1e8042759810a8de9ddbd390 P a198d9909afc4a1ca15b39b362477d26: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:40.165271 15804 tablet_bootstrap.cc:492] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01: No bootstrap required, opened a new log
I20260808 07:01:40.165271 15803 tablet_bootstrap.cc:492] T 10dc7e6a1e8042759810a8de9ddbd390 P f58811b83a68475599c2a11167f14492: No bootstrap required, opened a new log
I20260808 07:01:40.165336 15804 ts_tablet_manager.cc:1397] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:01:40.165398 15802 tablet_bootstrap.cc:492] T 10dc7e6a1e8042759810a8de9ddbd390 P a198d9909afc4a1ca15b39b362477d26: No bootstrap required, opened a new log
I20260808 07:01:40.165336 15803 ts_tablet_manager.cc:1397] T 10dc7e6a1e8042759810a8de9ddbd390 P f58811b83a68475599c2a11167f14492: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:01:40.165465 15802 ts_tablet_manager.cc:1397] T 10dc7e6a1e8042759810a8de9ddbd390 P a198d9909afc4a1ca15b39b362477d26: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:01:40.165521 15804 raft_consensus.cc:348] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:40.165529 15803 raft_consensus.cc:348] T 10dc7e6a1e8042759810a8de9ddbd390 P f58811b83a68475599c2a11167f14492 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:40.165611 15804 raft_consensus.cc:374] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:40.165611 15803 raft_consensus.cc:374] T 10dc7e6a1e8042759810a8de9ddbd390 P f58811b83a68475599c2a11167f14492 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:40.165634 15803 raft_consensus.cc:729] T 10dc7e6a1e8042759810a8de9ddbd390 P f58811b83a68475599c2a11167f14492 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f58811b83a68475599c2a11167f14492, State: Initialized, Role: FOLLOWER
I20260808 07:01:40.165634 15804 raft_consensus.cc:729] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1fa47d74bcb467584b6eb9044885f01, State: Initialized, Role: FOLLOWER
I20260808 07:01:40.165642 15802 raft_consensus.cc:348] T 10dc7e6a1e8042759810a8de9ddbd390 P a198d9909afc4a1ca15b39b362477d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:40.165681 15803 consensus_queue.cc:260] T 10dc7e6a1e8042759810a8de9ddbd390 P f58811b83a68475599c2a11167f14492 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:40.165681 15804 consensus_queue.cc:260] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:40.165727 15802 raft_consensus.cc:374] T 10dc7e6a1e8042759810a8de9ddbd390 P a198d9909afc4a1ca15b39b362477d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:40.165759 15802 raft_consensus.cc:729] T 10dc7e6a1e8042759810a8de9ddbd390 P a198d9909afc4a1ca15b39b362477d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a198d9909afc4a1ca15b39b362477d26, State: Initialized, Role: FOLLOWER
I20260808 07:01:40.165772 15803 ts_tablet_manager.cc:1428] T 10dc7e6a1e8042759810a8de9ddbd390 P f58811b83a68475599c2a11167f14492: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:01:40.165772 15804 ts_tablet_manager.cc:1428] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:01:40.165800 15802 consensus_queue.cc:260] T 10dc7e6a1e8042759810a8de9ddbd390 P a198d9909afc4a1ca15b39b362477d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:40.165880 15802 ts_tablet_manager.cc:1428] T 10dc7e6a1e8042759810a8de9ddbd390 P a198d9909afc4a1ca15b39b362477d26: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:40.165889 15822 tablet_replica.cc:439] T 10dc7e6a1e8042759810a8de9ddbd390 P f58811b83a68475599c2a11167f14492: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } } }
I20260808 07:01:40.165961 15822 tablet_replica.cc:442] T 10dc7e6a1e8042759810a8de9ddbd390 P f58811b83a68475599c2a11167f14492: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:01:40.165948 15809 tablet_replica.cc:439] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } } }
I20260808 07:01:40.165971 15810 tablet_replica.cc:439] T 10dc7e6a1e8042759810a8de9ddbd390 P a198d9909afc4a1ca15b39b362477d26: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } } }
I20260808 07:01:40.166045 15810 tablet_replica.cc:442] T 10dc7e6a1e8042759810a8de9ddbd390 P a198d9909afc4a1ca15b39b362477d26: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:01:40.166043 15809 tablet_replica.cc:442] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 07:01:40.274436 15491 tablet.cc:2369] T 10dc7e6a1e8042759810a8de9ddbd390 P f58811b83a68475599c2a11167f14492: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:01:40.294885 15809 raft_consensus.cc:482] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:01:40.295009 15809 raft_consensus.cc:504] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:40.295441 15809 leader_election.cc:288] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f58811b83a68475599c2a11167f14492 (127.13.207.1:32835), a198d9909afc4a1ca15b39b362477d26 (127.13.207.3:33337)
I20260808 07:01:40.299281 15801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:40.295730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:44797 (local address 127.13.207.3:33337)
0808 07:01:40.295783 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:40.295785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:40.295961 (+ 176us) server_negotiation.cc:407] Connection header received
0808 07:01:40.296060 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:40.296061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:40.296074 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:40.296142 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:40.296335 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.296840 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.298152 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.298831 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.298928 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:40.298939 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:40.299151 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 07:01:40.299194 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:01:40.299297 15843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:40.295669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:33337 (local address 127.13.207.2:44797)
0808 07:01:40.295799 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:40.295818 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:40.295959 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:40.296160 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:40.296161 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:40.296163 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:40.296255 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:40.296256 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.296950 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.296951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:40.298023 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:40.298026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.299002 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.299003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:40.299063 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:40.299067 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:01:40.299145 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:01:40.299199 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":44,"thread_start_us":65,"threads_started":1}
I20260808 07:01:40.299376 15800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:40.295685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:41287 (local address 127.13.207.1:32835)
0808 07:01:40.295764 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:40.295766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:40.295915 (+ 149us) server_negotiation.cc:407] Connection header received
0808 07:01:40.295954 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:40.295956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:40.295976 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:40.296046 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:40.296261 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.296824 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.298189 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.298834 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.298928 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:40.298939 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:40.299174 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 07:01:40.299288 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:01:40.299469 15842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:40.295579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:32835 (local address 127.13.207.2:41287)
0808 07:01:40.295760 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:40.295775 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:40.295864 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:40.296060 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:40.296061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:40.296064 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:40.296166 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:40.296169 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.297001 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.297002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:40.298032 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:40.298048 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.298989 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.298990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:40.299058 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:40.299063 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:01:40.299164 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 07:01:40.299288 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":6,"thread_start_us":66,"threads_started":1}
I20260808 07:01:40.299688 15730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10dc7e6a1e8042759810a8de9ddbd390" candidate_uuid: "e1fa47d74bcb467584b6eb9044885f01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a198d9909afc4a1ca15b39b362477d26" is_pre_election: true
I20260808 07:01:40.299811 15730 raft_consensus.cc:2454] T 10dc7e6a1e8042759810a8de9ddbd390 P a198d9909afc4a1ca15b39b362477d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1fa47d74bcb467584b6eb9044885f01 in term 0.
I20260808 07:01:40.299870 15443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10dc7e6a1e8042759810a8de9ddbd390" candidate_uuid: "e1fa47d74bcb467584b6eb9044885f01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f58811b83a68475599c2a11167f14492" is_pre_election: true
I20260808 07:01:40.300043 15443 raft_consensus.cc:2454] T 10dc7e6a1e8042759810a8de9ddbd390 P f58811b83a68475599c2a11167f14492 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1fa47d74bcb467584b6eb9044885f01 in term 0.
I20260808 07:01:40.300024 15518 leader_election.cc:302] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a198d9909afc4a1ca15b39b362477d26, e1fa47d74bcb467584b6eb9044885f01; no voters:
I20260808 07:01:40.300174 15809 raft_consensus.cc:2781] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:01:40.300248 15809 raft_consensus.cc:482] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:01:40.300287 15809 raft_consensus.cc:3037] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:40.301095 15809 raft_consensus.cc:504] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:40.301293 15809 leader_election.cc:288] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01 [CANDIDATE]: Term 1 election: Requested vote from peers f58811b83a68475599c2a11167f14492 (127.13.207.1:32835), a198d9909afc4a1ca15b39b362477d26 (127.13.207.3:33337)
I20260808 07:01:40.301493 15730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10dc7e6a1e8042759810a8de9ddbd390" candidate_uuid: "e1fa47d74bcb467584b6eb9044885f01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a198d9909afc4a1ca15b39b362477d26"
I20260808 07:01:40.301551 15443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10dc7e6a1e8042759810a8de9ddbd390" candidate_uuid: "e1fa47d74bcb467584b6eb9044885f01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f58811b83a68475599c2a11167f14492"
I20260808 07:01:40.301608 15730 raft_consensus.cc:3037] T 10dc7e6a1e8042759810a8de9ddbd390 P a198d9909afc4a1ca15b39b362477d26 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:40.301637 15443 raft_consensus.cc:3037] T 10dc7e6a1e8042759810a8de9ddbd390 P f58811b83a68475599c2a11167f14492 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:40.302400 15730 raft_consensus.cc:2454] T 10dc7e6a1e8042759810a8de9ddbd390 P a198d9909afc4a1ca15b39b362477d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1fa47d74bcb467584b6eb9044885f01 in term 1.
I20260808 07:01:40.302709 15443 raft_consensus.cc:2454] T 10dc7e6a1e8042759810a8de9ddbd390 P f58811b83a68475599c2a11167f14492 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1fa47d74bcb467584b6eb9044885f01 in term 1.
I20260808 07:01:40.302914 15519 leader_election.cc:302] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e1fa47d74bcb467584b6eb9044885f01, f58811b83a68475599c2a11167f14492; no voters:
I20260808 07:01:40.303048 15809 raft_consensus.cc:2781] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:40.303254 15809 raft_consensus.cc:686] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01 [term 1 LEADER]: Becoming Leader. State: Replica: e1fa47d74bcb467584b6eb9044885f01, State: Running, Role: LEADER
I20260808 07:01:40.303346 15809 consensus_queue.cc:237] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } }
I20260808 07:01:40.303826 15844 tablet_replica.cc:439] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01: TxnStatusTablet state changed. Reason: New leader e1fa47d74bcb467584b6eb9044885f01. Latest consensus state: current_term: 1 leader_uuid: "e1fa47d74bcb467584b6eb9044885f01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } } }
I20260808 07:01:40.304075 15844 tablet_replica.cc:442] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:01:40.304188 15282 catalog_manager.cc:5495] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1fa47d74bcb467584b6eb9044885f01 (127.13.207.2). New cstate: current_term: 1 leader_uuid: "e1fa47d74bcb467584b6eb9044885f01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } health_report { overall_health: UNKNOWN } } }
I20260808 07:01:40.304342 15846 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 07:01:40.304518 15846 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 07:01:40.318639 15837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:40.315742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:39515 (local address 127.0.0.1:51554)
0808 07:01:40.315832 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:40.315847 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:40.315926 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:40.316195 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:40.316196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:40.316198 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:01:40.316255 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:40.316257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.316991 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.316992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:40.317624 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:40.317626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.318023 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.318024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:40.318098 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:40.318120 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:01:40.318437 (+ 317us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:01:40.318448 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 07:01:40.318509 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 07:01:40.318564 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 07:01:40.318627 15799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:40.315865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51554 (local address 127.13.207.2:39515)
0808 07:01:40.315932 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:40.315934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:40.315945 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:01:40.316078 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:40.316079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:40.316107 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:40.316188 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:01:40.316327 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.316888 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.317721 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.317936 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.318025 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:40.318238 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:01:40.318351 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:01:40.318447 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:40.318512 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 07:01:40.318536 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:01:40.320103 15730 raft_consensus.cc:1261] T 10dc7e6a1e8042759810a8de9ddbd390 P a198d9909afc4a1ca15b39b362477d26 [term 1 FOLLOWER]: Refusing update from remote peer e1fa47d74bcb467584b6eb9044885f01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:01:40.320266 15443 raft_consensus.cc:1261] T 10dc7e6a1e8042759810a8de9ddbd390 P f58811b83a68475599c2a11167f14492 [term 1 FOLLOWER]: Refusing update from remote peer e1fa47d74bcb467584b6eb9044885f01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:01:40.320421 15809 consensus_queue.cc:1035] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:01:40.320657 15809 consensus_queue.cc:1035] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:01:40.321151 15810 tablet_replica.cc:439] T 10dc7e6a1e8042759810a8de9ddbd390 P a198d9909afc4a1ca15b39b362477d26: TxnStatusTablet state changed. Reason: New leader e1fa47d74bcb467584b6eb9044885f01. Latest consensus state: current_term: 1 leader_uuid: "e1fa47d74bcb467584b6eb9044885f01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } } }
I20260808 07:01:40.321255 15810 tablet_replica.cc:442] T 10dc7e6a1e8042759810a8de9ddbd390 P a198d9909afc4a1ca15b39b362477d26: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:01:40.321666 15822 tablet_replica.cc:439] T 10dc7e6a1e8042759810a8de9ddbd390 P f58811b83a68475599c2a11167f14492: TxnStatusTablet state changed. Reason: New leader e1fa47d74bcb467584b6eb9044885f01. Latest consensus state: current_term: 1 leader_uuid: "e1fa47d74bcb467584b6eb9044885f01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } } }
I20260808 07:01:40.321772 15822 tablet_replica.cc:442] T 10dc7e6a1e8042759810a8de9ddbd390 P f58811b83a68475599c2a11167f14492: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:01:40.322170 15817 mvcc.cc:205] Tried to move back new op lower bound from 7316162561308995584 to 7316162561243332608. Current Snapshot: MvccSnapshot[applied={T|T < 7316162561308995584}]
I20260808 07:01:40.322284 15809 tablet_replica.cc:439] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e1fa47d74bcb467584b6eb9044885f01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } } }
I20260808 07:01:40.322355 15809 tablet_replica.cc:442] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:01:40.322407 15809 tablet_replica.cc:439] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e1fa47d74bcb467584b6eb9044885f01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } } }
I20260808 07:01:40.322448 15809 tablet_replica.cc:442] T 10dc7e6a1e8042759810a8de9ddbd390 P e1fa47d74bcb467584b6eb9044885f01: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:01:40.322866 15816 mvcc.cc:205] Tried to move back new op lower bound from 7316162561308995584 to 7316162561243332608. Current Snapshot: MvccSnapshot[applied={T|T < 7316162561308995584}]
I20260808 07:01:40.322968 15810 tablet_replica.cc:439] T 10dc7e6a1e8042759810a8de9ddbd390 P a198d9909afc4a1ca15b39b362477d26: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e1fa47d74bcb467584b6eb9044885f01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } } }
I20260808 07:01:40.323133 15810 tablet_replica.cc:442] T 10dc7e6a1e8042759810a8de9ddbd390 P a198d9909afc4a1ca15b39b362477d26: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:01:40.323312 15815 mvcc.cc:205] Tried to move back new op lower bound from 7316162561308995584 to 7316162561243332608. Current Snapshot: MvccSnapshot[applied={T|T < 7316162561308995584}]
I20260808 07:01:40.323432 15822 tablet_replica.cc:439] T 10dc7e6a1e8042759810a8de9ddbd390 P f58811b83a68475599c2a11167f14492: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e1fa47d74bcb467584b6eb9044885f01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58811b83a68475599c2a11167f14492" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32835 } } peers { permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 } } peers { permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 } } }
I20260808 07:01:40.323503 15822 tablet_replica.cc:442] T 10dc7e6a1e8042759810a8de9ddbd390 P f58811b83a68475599c2a11167f14492: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:01:40.330533 15799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:40.327904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:49257 (local address 127.13.207.2:39515)
0808 07:01:40.327955 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:40.327956 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:40.327974 (+ 18us) server_negotiation.cc:407] Connection header received
0808 07:01:40.328066 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:40.328066 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:40.328076 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:40.328121 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:01:40.328274 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.328869 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.329725 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.329965 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.330044 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:40.330253 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:01:40.330291 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:01:40.330359 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:40.330424 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 07:01:40.330447 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 07:01:40.330579 15854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:40.327720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:39515 (local address 127.13.207.1:49257)
0808 07:01:40.327912 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:40.327926 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:40.327994 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:40.328141 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:40.328142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:40.328145 (+ 3us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:01:40.328209 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:40.328210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.328952 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.328953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:40.329560 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:40.329562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.330047 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.330048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:40.330112 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:40.330136 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:01:40.330362 (+ 226us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:01:40.330365 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:01:40.330420 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 07:01:40.330478 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":107,"threads_started":1}
I20260808 07:01:40.333110 15730 raft_consensus.cc:1261] T 346171a1a8fd43519d1585336cc70a2c P a198d9909afc4a1ca15b39b362477d26 [term 1 FOLLOWER]: Refusing update from remote peer f58811b83a68475599c2a11167f14492: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:01:40.333294 15586 raft_consensus.cc:1261] T 346171a1a8fd43519d1585336cc70a2c P e1fa47d74bcb467584b6eb9044885f01 [term 1 FOLLOWER]: Refusing update from remote peer f58811b83a68475599c2a11167f14492: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:01:40.333559 15822 consensus_queue.cc:1035] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a198d9909afc4a1ca15b39b362477d26" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:01:40.333743 15808 consensus_queue.cc:1035] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1fa47d74bcb467584b6eb9044885f01" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:01:40.345504 15858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:40.341190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:32835 (local address 127.13.207.2:40059)
0808 07:01:40.341367 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:40.341387 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:40.341454 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:40.341596 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:40.341596 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:40.341597 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:01:40.341663 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:40.341664 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.342705 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.342706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:40.344436 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:40.344439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.344849 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.344850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:40.344930 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:40.344952 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:01:40.345231 (+ 279us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:01:40.345234 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:01:40.345293 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 07:01:40.345348 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
I20260808 07:01:40.345846 15800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:40.341234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:40059 (local address 127.13.207.1:32835)
0808 07:01:40.341285 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:40.341287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:40.341465 (+ 178us) server_negotiation.cc:407] Connection header received
0808 07:01:40.341521 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:40.341522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:40.341533 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:40.341594 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:01:40.341704 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.342567 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.344544 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.344762 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.344829 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:40.345106 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:01:40.345152 (+ 46us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:01:40.345216 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:40.345757 (+ 541us) server_negotiation.cc:299] Negotiation successful
0808 07:01:40.345793 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:01:40.358991 15281 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:01:40.359130 15281 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:01:40.360014 15281 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fbdd7ddae2494eff8bd05f069aae8f27: Sending DeleteTablet for 3 replicas of tablet 346171a1a8fd43519d1585336cc70a2c
I20260808 07:01:40.360373 15422 tablet_service.cc:1514] Processing DeleteTablet for tablet 346171a1a8fd43519d1585336cc70a2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:40 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:56592
I20260808 07:01:40.360529 15825 tablet_replica.cc:330] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492: stopping tablet replica
I20260808 07:01:40.360514 15709 tablet_service.cc:1514] Processing DeleteTablet for tablet 346171a1a8fd43519d1585336cc70a2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:40 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:40820
I20260808 07:01:40.360524 15565 tablet_service.cc:1514] Processing DeleteTablet for tablet 346171a1a8fd43519d1585336cc70a2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:40 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:51540
I20260808 07:01:40.360606 15825 raft_consensus.cc:2229] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:01:40.360664 15826 tablet_replica.cc:330] T 346171a1a8fd43519d1585336cc70a2c P a198d9909afc4a1ca15b39b362477d26: stopping tablet replica
I20260808 07:01:40.360692 15825 raft_consensus.cc:2258] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:01:40.360733 15826 raft_consensus.cc:2229] T 346171a1a8fd43519d1585336cc70a2c P a198d9909afc4a1ca15b39b362477d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:01:40.360778 15826 raft_consensus.cc:2258] T 346171a1a8fd43519d1585336cc70a2c P a198d9909afc4a1ca15b39b362477d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:01:40.360857 15827 tablet_replica.cc:330] T 346171a1a8fd43519d1585336cc70a2c P e1fa47d74bcb467584b6eb9044885f01: stopping tablet replica
I20260808 07:01:40.360929 15827 raft_consensus.cc:2229] T 346171a1a8fd43519d1585336cc70a2c P e1fa47d74bcb467584b6eb9044885f01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:01:40.360985 15827 raft_consensus.cc:2258] T 346171a1a8fd43519d1585336cc70a2c P e1fa47d74bcb467584b6eb9044885f01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:01:40.361361 15827 ts_tablet_manager.cc:1905] T 346171a1a8fd43519d1585336cc70a2c P e1fa47d74bcb467584b6eb9044885f01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:01:40.361716 15826 ts_tablet_manager.cc:1905] T 346171a1a8fd43519d1585336cc70a2c P a198d9909afc4a1ca15b39b362477d26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:01:40.361850 15825 ts_tablet_manager.cc:1905] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:01:40.363813 15827 ts_tablet_manager.cc:1918] T 346171a1a8fd43519d1585336cc70a2c P e1fa47d74bcb467584b6eb9044885f01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:01:40.363883 15827 log.cc:1196] T 346171a1a8fd43519d1585336cc70a2c P e1fa47d74bcb467584b6eb9044885f01: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-1/wal/wals/346171a1a8fd43519d1585336cc70a2c
I20260808 07:01:40.364144 15826 ts_tablet_manager.cc:1918] T 346171a1a8fd43519d1585336cc70a2c P a198d9909afc4a1ca15b39b362477d26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:01:40.364146 15825 ts_tablet_manager.cc:1918] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:01:40.364215 15826 log.cc:1196] T 346171a1a8fd43519d1585336cc70a2c P a198d9909afc4a1ca15b39b362477d26: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/346171a1a8fd43519d1585336cc70a2c
I20260808 07:01:40.364215 15825 log.cc:1196] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172493058068-14140-0/minicluster-data/ts-0/wal/wals/346171a1a8fd43519d1585336cc70a2c
I20260808 07:01:40.364272 15827 ts_tablet_manager.cc:1939] T 346171a1a8fd43519d1585336cc70a2c P e1fa47d74bcb467584b6eb9044885f01: Deleting consensus metadata
I20260808 07:01:40.364485 15826 ts_tablet_manager.cc:1939] T 346171a1a8fd43519d1585336cc70a2c P a198d9909afc4a1ca15b39b362477d26: Deleting consensus metadata
I20260808 07:01:40.364491 15825 ts_tablet_manager.cc:1939] T 346171a1a8fd43519d1585336cc70a2c P f58811b83a68475599c2a11167f14492: Deleting consensus metadata
I20260808 07:01:40.364751 15267 catalog_manager.cc:4841] TS e1fa47d74bcb467584b6eb9044885f01 (127.13.207.2:39515): tablet 346171a1a8fd43519d1585336cc70a2c (table test-table [id=6d445185f75a40cd9968a9395cb3686c]) successfully deleted
I20260808 07:01:40.364858 15269 catalog_manager.cc:4841] TS f58811b83a68475599c2a11167f14492 (127.13.207.1:32835): tablet 346171a1a8fd43519d1585336cc70a2c (table test-table [id=6d445185f75a40cd9968a9395cb3686c]) successfully deleted
I20260808 07:01:40.364998 15268 catalog_manager.cc:4841] TS a198d9909afc4a1ca15b39b362477d26 (127.13.207.3:33337): tablet 346171a1a8fd43519d1585336cc70a2c (table test-table [id=6d445185f75a40cd9968a9395cb3686c]) successfully deleted
Aug 08 07:01:40 dist-test-slave-8tdl krb5kdc[15236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172499, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.207.1@KRBTEST.COM
I20260808 07:01:40.374619 15800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:40.363628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56622 (local address 127.13.207.1:32835)
0808 07:01:40.363709 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:40.363711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:40.364104 (+ 393us) server_negotiation.cc:407] Connection header received
0808 07:01:40.364232 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:40.364233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:40.364251 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:40.364287 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:40.365176 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.365694 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.366946 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.367138 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.367208 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:40.371189 (+ 3981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:40.371203 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:40.371204 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:40.371214 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:40.373162 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:40.373596 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:40.373603 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:40.373603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:40.373671 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:40.373902 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:40.373903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:40.373904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:40.374141 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:40.374339 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:40.374385 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 07:01:40.374491 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 07:01:40.374965 15474 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56622
I20260808 07:01:40.384251 15332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:40.376103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44416 (local address 127.13.207.62:40015)
0808 07:01:40.376172 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:40.376175 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:40.376261 (+ 86us) server_negotiation.cc:407] Connection header received
0808 07:01:40.376314 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:40.376316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:40.376327 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:40.376373 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:40.377079 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.377740 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.379020 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:40.379287 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:40.379378 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:40.380947 (+ 1569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:40.380948 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:40.380949 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:40.380956 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:40.382857 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:40.383317 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:40.383318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:40.383318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:40.383432 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:40.383692 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:40.383693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:40.383693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:40.383841 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:40.383932 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:40.384088 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 07:01:40.384176 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260808 07:01:40.384742 15281 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44416
I20260808 07:01:40.385667 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 15355
I20260808 07:01:40.395274 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 15499
I20260808 07:01:40.401595 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 15643
I20260808 07:01:40.408318 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 15250
2026-08-08T07:01:40Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2608 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:01:40 dist-test-slave-8tdl krb5kdc[15866](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:01:40 dist-test-slave-8tdl krb5kdc[15866](info): set up 2 sockets
Aug 08 07:01:40 dist-test-slave-8tdl krb5kdc[15866](info): commencing operation
krb5kdc: starting...
W20260808 07:01:41.515756 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:01:41 dist-test-slave-8tdl krb5kdc[15866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:01:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:01:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:01:41.659770 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:38167
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:35327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:38167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:41.749800 15880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:41.750097 15880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:41.750140 15880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:41.751570 15880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:01:41.751665 15880 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 07:01:41.751686 15880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:41.751708 15880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:41.751739 15880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:01:41.751768 15880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:01:41.753888 15880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:35327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:38167
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:38167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:41.754256 15880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:41.754546 15880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:41.760393 15889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:41.760407 15887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:41.760393 15886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:41.761122 15880 server_base.cc:1034] running on GCE node
I20260808 07:01:41.761489 15880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:41.761852 15880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:41.763130 15880 hybrid_clock.cc:648] HybridClock initialized: now 1786172501763091 us; error 58 us; skew 500 ppm
Aug 08 07:01:41 dist-test-slave-8tdl krb5kdc[15866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172501, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:41.766359 15880 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:01:41.767017 15880 webserver.cc:460] Webserver started at http://127.13.207.62:41035/ using document root <none> and password file <none>
I20260808 07:01:41.767290 15880 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:41.767365 15880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:41.767529 15880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:41.768769 15880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "be83e223859f4e59b15d2d5421664830"
format_stamp: "Formatted at 2026-08-08 07:01:41 on dist-test-slave-8tdl"
server_key: "9dd9fe31011fc561479259b95481a913"
server_key_iv: "0c8f519e3023bc4884bf15e1e371bcb5"
server_key_version: "encryptionkey@0"
I20260808 07:01:41.769199 15880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "be83e223859f4e59b15d2d5421664830"
format_stamp: "Formatted at 2026-08-08 07:01:41 on dist-test-slave-8tdl"
server_key: "9dd9fe31011fc561479259b95481a913"
server_key_iv: "0c8f519e3023bc4884bf15e1e371bcb5"
server_key_version: "encryptionkey@0"
I20260808 07:01:41.771072 15880 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 07:01:41.772233 15896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:41.772483 15880 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:41.772645 15880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "be83e223859f4e59b15d2d5421664830"
format_stamp: "Formatted at 2026-08-08 07:01:41 on dist-test-slave-8tdl"
server_key: "9dd9fe31011fc561479259b95481a913"
server_key_iv: "0c8f519e3023bc4884bf15e1e371bcb5"
server_key_version: "encryptionkey@0"
I20260808 07:01:41.772761 15880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:41.802400 15880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:41.805073 15880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:41.805274 15880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:41.810843 15880 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:38167
I20260808 07:01:41.810897 15958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:38167 every 8 connection(s)
I20260808 07:01:41.811283 15880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:01:41.811962 15959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:41.814481 15959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830: Bootstrap starting.
I20260808 07:01:41.815240 15959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:41.815577 15959 log.cc:824] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:41.816440 15959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830: No bootstrap required, opened a new log
I20260808 07:01:41.816474 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 15880
I20260808 07:01:41.816603 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:01:41.816777 14140 external_mini_cluster.cc:1428] Setting key b7f3d41b2b35ef4b6db873937eab8339
I20260808 07:01:41.819093 15959 raft_consensus.cc:348] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be83e223859f4e59b15d2d5421664830" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38167 } }
I20260808 07:01:41.819298 15959 raft_consensus.cc:374] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:41.819340 15959 raft_consensus.cc:729] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be83e223859f4e59b15d2d5421664830, State: Initialized, Role: FOLLOWER
I20260808 07:01:41.819478 15959 consensus_queue.cc:260] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be83e223859f4e59b15d2d5421664830" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38167 } }
I20260808 07:01:41.819571 15959 raft_consensus.cc:388] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:01:41.819624 15959 raft_consensus.cc:482] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:01:41.819702 15959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:41.820395 15959 raft_consensus.cc:504] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be83e223859f4e59b15d2d5421664830" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38167 } }
I20260808 07:01:41.820529 15959 leader_election.cc:302] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be83e223859f4e59b15d2d5421664830; no voters:
I20260808 07:01:41.820767 15959 leader_election.cc:288] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:01:41.820807 15964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:41.820927 15964 raft_consensus.cc:686] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830 [term 1 LEADER]: Becoming Leader. State: Replica: be83e223859f4e59b15d2d5421664830, State: Running, Role: LEADER
I20260808 07:01:41.821018 15964 consensus_queue.cc:237] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be83e223859f4e59b15d2d5421664830" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38167 } }
I20260808 07:01:41.821082 15959 sys_catalog.cc:564] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:01:41.821379 15964 sys_catalog.cc:455] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be83e223859f4e59b15d2d5421664830. Latest consensus state: current_term: 1 leader_uuid: "be83e223859f4e59b15d2d5421664830" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be83e223859f4e59b15d2d5421664830" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38167 } } }
I20260808 07:01:41.821475 15964 sys_catalog.cc:458] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830 [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:41.821391 15966 sys_catalog.cc:455] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be83e223859f4e59b15d2d5421664830" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be83e223859f4e59b15d2d5421664830" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38167 } } }
I20260808 07:01:41.821691 15966 sys_catalog.cc:458] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830 [sys.catalog]: This master's current role is: LEADER
W20260808 07:01:41.823370 15979 catalog_manager.cc:1558] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:01:41.823465 15979 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:01:41.824046 15969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:01:41.824268 15969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 07:01:41 dist-test-slave-8tdl krb5kdc[15866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:41.825879 15969 catalog_manager.cc:1347] Generated new cluster ID: fdb299d3023d43dc82587768adcff293
I20260808 07:01:41.825937 15969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:01:41.830431 15963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:41.817361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40830 (local address 127.13.207.62:38167)
0808 07:01:41.817580 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:41.817591 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:41.817625 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:01:41.819087 (+ 1462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:41.819096 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:41.819173 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:41.819367 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:41.820215 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:41.821484 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:41.822367 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:41.822926 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:41.823508 (+ 582us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:41.826133 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:41.826137 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:41.826138 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:41.826151 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:41.828456 (+ 2305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:41.828958 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:41.828962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:41.828962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:41.829116 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:41.829381 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:41.829382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:41.829382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:41.829721 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:41.829872 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:41.830049 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 07:01:41.830205 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":106,"threads_started":1}
I20260808 07:01:41.839195 15969 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:01:41.840009 15969 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:01:41.845104 15969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830: Generated new TSK 0
I20260808 07:01:41.845403 15969 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
I20260808 07:01:41.884001 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:38167
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:35327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:41.979156 15985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:41.979449 15985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:41.979481 15985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:41.980813 15985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:41.980880 15985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:41.980906 15985 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 07:01:41.980917 15985 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:01:41.980966 15985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:01:41.982800 15985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:35327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.207.62:38167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:41.983100 15985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:41.983400 15985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:41.988936 15993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:41.988974 15990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:41.989104 15985 server_base.cc:1034] running on GCE node
W20260808 07:01:41.988974 15991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:41.989456 15985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:41.989753 15985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:41.990988 15985 hybrid_clock.cc:648] HybridClock initialized: now 1786172501990957 us; error 59 us; skew 500 ppm
Aug 08 07:01:41 dist-test-slave-8tdl krb5kdc[15866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172501, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:41.994292 15985 init.cc:377] Logged in from keytab as kudu/127.13.207.1@KRBTEST.COM (short username kudu)
I20260808 07:01:41.994997 15985 webserver.cc:460] Webserver started at http://127.13.207.1:44263/ using document root <none> and password file <none>
I20260808 07:01:41.995309 15985 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:41.995365 15985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:41.995590 15985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:41.996897 15985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "5568508efd67416d8c5a791cef3fd44f"
format_stamp: "Formatted at 2026-08-08 07:01:41 on dist-test-slave-8tdl"
server_key: "6f83e8d149916a8f4199b627c2b6ec4c"
server_key_iv: "8579d484f2613f52e327f730b977e24f"
server_key_version: "encryptionkey@0"
I20260808 07:01:41.997296 15985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "5568508efd67416d8c5a791cef3fd44f"
format_stamp: "Formatted at 2026-08-08 07:01:41 on dist-test-slave-8tdl"
server_key: "6f83e8d149916a8f4199b627c2b6ec4c"
server_key_iv: "8579d484f2613f52e327f730b977e24f"
server_key_version: "encryptionkey@0"
I20260808 07:01:41.998873 15985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 07:01:41.999959 16000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:42.000480 15985 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:42.000622 15985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "5568508efd67416d8c5a791cef3fd44f"
format_stamp: "Formatted at 2026-08-08 07:01:41 on dist-test-slave-8tdl"
server_key: "6f83e8d149916a8f4199b627c2b6ec4c"
server_key_iv: "8579d484f2613f52e327f730b977e24f"
server_key_version: "encryptionkey@0"
I20260808 07:01:42.001171 15985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:42.014485 15985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:42.017382 15985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:42.017656 15985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:42.018497 15985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:42.018590 15985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:42.018636 15985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:42.018680 15985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:01:42 dist-test-slave-8tdl krb5kdc[15866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172501, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:42.028501 15985 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:36077
I20260808 07:01:42.028577 16118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:36077 every 8 connection(s)
I20260808 07:01:42.029069 15985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:01:42.031719 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 15985
I20260808 07:01:42.031831 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:01:42.031833 15963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.019238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:54995 (local address 127.13.207.62:38167)
0808 07:01:42.019305 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:42.019308 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:42.019551 (+ 243us) server_negotiation.cc:407] Connection header received
0808 07:01:42.020340 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:42.020342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:42.020359 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:42.020428 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:42.021201 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.022362 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.024007 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.024255 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.024364 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.027598 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:42.027601 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:42.027601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:42.027612 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:42.029583 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:42.030113 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:42.030114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:42.030114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:42.030187 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:42.030623 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:42.030624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:42.030624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:42.030868 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:42.031003 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:42.031551 (+ 548us) server_negotiation.cc:299] Negotiation successful
0808 07:01:42.031690 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:01:42.031826 16009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.019173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38167 (local address 127.13.207.1:54995)
0808 07:01:42.019381 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:42.019422 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:42.020049 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:42.020513 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:42.020517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:42.020804 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:42.021076 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.021083 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.022545 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.022548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.023823 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.023830 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.024382 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.024383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.024565 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.025186 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:42.025195 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:42.027441 (+ 2246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:42.029703 (+ 2262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:42.029710 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:42.029712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:42.029985 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:42.030402 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:42.030403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:42.030403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:42.030527 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:42.031129 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:42.031136 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:42.031280 (+ 144us) client_negotiation.cc:769] Sending connection context
0808 07:01:42.031390 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 07:01:42.031579 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":112,"threads_started":1}
I20260808 07:01:42.032085 14140 external_mini_cluster.cc:1428] Setting key 45a9c2fb63bb40a56bb39c0de89cc666
I20260808 07:01:42.032894 16119 heartbeater.cc:347] Connected to a master server at 127.13.207.62:38167
I20260808 07:01:42.035035 16119 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:42.035284 16119 heartbeater.cc:511] Master 127.13.207.62:38167 requested a full tablet report, sending...
I20260808 07:01:42.035995 15912 ts_manager.cc:194] Registered new tserver with Master: 5568508efd67416d8c5a791cef3fd44f (127.13.207.1:36077)
I20260808 07:01:42.036725 15912 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:54995
I20260808 07:01:42.042208 15963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.034238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:45861 (local address 127.13.207.62:38167)
0808 07:01:42.034292 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:42.034294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:42.035712 (+ 1418us) server_negotiation.cc:407] Connection header received
0808 07:01:42.035736 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:42.035737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:42.035748 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:42.035795 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:42.036355 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.036927 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.037931 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.038140 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.038229 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.039376 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:42.039378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:42.039378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:42.039387 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:42.040924 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:42.041329 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:42.041331 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:42.041331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:42.041400 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:42.041616 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:42.041617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:42.041617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:42.041748 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:42.041837 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:42.041996 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 07:01:42.042101 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:01:42.042223 16125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.034123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38167 (local address 127.13.207.1:45861)
0808 07:01:42.035524 (+ 1401us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:42.035535 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:42.035625 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:42.035858 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:42.035860 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:42.036147 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:42.036228 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.036231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.037058 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.037058 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.037824 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.037828 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.038220 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.038221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.038288 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.038897 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:42.038903 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:42.039246 (+ 343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:42.041045 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:42.041046 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:42.041047 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:42.041249 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:42.041473 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:42.041474 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:42.041474 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:42.041548 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:42.041819 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:42.041820 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:42.041875 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 07:01:42.041967 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 07:01:42.042099 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1372,"thread_start_us":85,"threads_started":1}
WARNING: no policy specified for kudu/127.13.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
I20260808 07:01:42.067152 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:38167
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:35327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:42.161334 16128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:42.161586 16128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:42.161615 16128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:42.162915 16128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:42.162976 16128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:42.163002 16128 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 07:01:42.163015 16128 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:01:42.163034 16128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:01:42.164703 16128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:35327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.207.62:38167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:42.164953 16128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:42.165179 16128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:42.170874 16133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:42.170878 16136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:42.170874 16134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:42.171669 16128 server_base.cc:1034] running on GCE node
I20260808 07:01:42.172029 16128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:42.172336 16128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:42.173532 16128 hybrid_clock.cc:648] HybridClock initialized: now 1786172502173488 us; error 63 us; skew 500 ppm
Aug 08 07:01:42 dist-test-slave-8tdl krb5kdc[15866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172502, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:42.176683 16128 init.cc:377] Logged in from keytab as kudu/127.13.207.2@KRBTEST.COM (short username kudu)
I20260808 07:01:42.177263 16128 webserver.cc:460] Webserver started at http://127.13.207.2:37587/ using document root <none> and password file <none>
I20260808 07:01:42.177534 16128 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:42.177606 16128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:42.177778 16128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:42.178996 16128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be"
format_stamp: "Formatted at 2026-08-08 07:01:42 on dist-test-slave-8tdl"
server_key: "62fcb2b54668e51c49ef406305477aa2"
server_key_iv: "52e7b29b0dd41dc98bdc3481e0a09191"
server_key_version: "encryptionkey@0"
I20260808 07:01:42.179374 16128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be"
format_stamp: "Formatted at 2026-08-08 07:01:42 on dist-test-slave-8tdl"
server_key: "62fcb2b54668e51c49ef406305477aa2"
server_key_iv: "52e7b29b0dd41dc98bdc3481e0a09191"
server_key_version: "encryptionkey@0"
I20260808 07:01:42.180929 16128 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:01:42.181905 16143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:42.182152 16128 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:42.182271 16128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be"
format_stamp: "Formatted at 2026-08-08 07:01:42 on dist-test-slave-8tdl"
server_key: "62fcb2b54668e51c49ef406305477aa2"
server_key_iv: "52e7b29b0dd41dc98bdc3481e0a09191"
server_key_version: "encryptionkey@0"
I20260808 07:01:42.182356 16128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:42.200206 16128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:42.202843 16128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:42.203056 16128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:42.203747 16128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:42.203819 16128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:42.203862 16128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:42.203877 16128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:01:42 dist-test-slave-8tdl krb5kdc[15866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172502, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:42.211586 16128 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:34543
I20260808 07:01:42.211680 16260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:34543 every 8 connection(s)
I20260808 07:01:42.211990 16128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:01:42.214443 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 16128
I20260808 07:01:42.214564 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:01:42.214800 14140 external_mini_cluster.cc:1428] Setting key 48d6989f6c42cf3663c56a492f6d5088
I20260808 07:01:42.214843 15963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.204417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:49883 (local address 127.13.207.62:38167)
0808 07:01:42.204499 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:42.204502 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:42.204669 (+ 167us) server_negotiation.cc:407] Connection header received
0808 07:01:42.205277 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:42.205279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:42.205292 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:42.205377 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:42.206136 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.207007 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.208088 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.208300 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.208393 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.211551 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:42.211553 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:42.211554 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:42.211564 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:42.213173 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:42.213674 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:42.213675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:42.213676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:42.213747 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:42.214009 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:42.214010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:42.214011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:42.214147 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:42.214230 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:42.214582 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 07:01:42.214664 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 07:01:42.215041 16154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.204344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38167 (local address 127.13.207.2:49883)
0808 07:01:42.204530 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:42.204573 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:42.205183 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:42.205448 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:42.205451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:42.205753 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:42.206028 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.206033 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.207123 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.207124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.207984 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.207986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.208408 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.208408 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.208538 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.209155 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:42.209168 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:42.211431 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:42.213304 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:42.213312 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:42.213314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:42.213585 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:42.213818 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:42.213819 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:42.213819 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:42.213931 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:42.214219 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:42.214225 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:42.214406 (+ 181us) client_negotiation.cc:769] Sending connection context
0808 07:01:42.214540 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 07:01:42.214767 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":108,"threads_started":1}
I20260808 07:01:42.215602 16261 heartbeater.cc:347] Connected to a master server at 127.13.207.62:38167
I20260808 07:01:42.215706 16261 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:42.215929 16261 heartbeater.cc:511] Master 127.13.207.62:38167 requested a full tablet report, sending...
I20260808 07:01:42.217530 15912 ts_manager.cc:194] Registered new tserver with Master: dd53f8d8a09a47d5b2ddcc7e05f098be (127.13.207.2:34543)
I20260808 07:01:42.217931 15912 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:49883
I20260808 07:01:42.225569 16267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.216694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38167 (local address 127.13.207.2:37527)
0808 07:01:42.218125 (+ 1431us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:42.218137 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:42.218210 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:42.218339 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:42.218340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:42.218557 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:42.218629 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.218631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.219418 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.219419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.220334 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.220338 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.220774 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.220775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.220981 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.221714 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:42.221719 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:42.222134 (+ 415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:42.224365 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:42.224366 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:42.224366 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:42.224560 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:42.224798 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:42.224799 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:42.224799 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:42.224857 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:42.225260 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:42.225261 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:42.225310 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 07:01:42.225369 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 07:01:42.225475 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1386,"thread_start_us":79,"threads_started":1}
I20260808 07:01:42.225646 15963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.216840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:37527 (local address 127.13.207.62:38167)
0808 07:01:42.216878 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:42.216880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:42.218211 (+ 1331us) server_negotiation.cc:407] Connection header received
0808 07:01:42.218263 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:42.218264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:42.218276 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:42.218336 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:42.218737 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.219270 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.220440 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.220688 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.220770 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.222233 (+ 1463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:42.222235 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:42.222236 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:42.222245 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:42.224245 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:42.224648 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:42.224650 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:42.224650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:42.224724 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:42.224925 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:42.224926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:42.224927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:42.225179 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:42.225266 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:42.225425 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 07:01:42.225518 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
WARNING: no policy specified for kudu/127.13.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
I20260808 07:01:42.252172 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:38167
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:35327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:42.342167 16270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:42.342469 16270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:42.342521 16270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:42.343918 16270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:42.344003 16270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:42.344041 16270 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 07:01:42.344071 16270 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:01:42.344110 16270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:01:42.346019 16270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:35327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.207.62:38167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:42.346339 16270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:42.346594 16270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:42.352214 16276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:42.352350 16270 server_base.cc:1034] running on GCE node
W20260808 07:01:42.352208 16279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:42.352208 16277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:42.352761 16270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:42.353067 16270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:42.354255 16270 hybrid_clock.cc:648] HybridClock initialized: now 1786172502354221 us; error 51 us; skew 500 ppm
Aug 08 07:01:42 dist-test-slave-8tdl krb5kdc[15866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172502, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:42.357818 16270 init.cc:377] Logged in from keytab as kudu/127.13.207.3@KRBTEST.COM (short username kudu)
I20260808 07:01:42.358404 16270 webserver.cc:460] Webserver started at http://127.13.207.3:35479/ using document root <none> and password file <none>
I20260808 07:01:42.358650 16270 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:42.358753 16270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:42.358912 16270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:42.360054 16270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "88e89354ac114e9b878e1be757114823"
format_stamp: "Formatted at 2026-08-08 07:01:42 on dist-test-slave-8tdl"
server_key: "f396646bdf3d18710fd22dfe45352011"
server_key_iv: "a8d06558509ca62cb0c76c8787f9cfd2"
server_key_version: "encryptionkey@0"
I20260808 07:01:42.360459 16270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "88e89354ac114e9b878e1be757114823"
format_stamp: "Formatted at 2026-08-08 07:01:42 on dist-test-slave-8tdl"
server_key: "f396646bdf3d18710fd22dfe45352011"
server_key_iv: "a8d06558509ca62cb0c76c8787f9cfd2"
server_key_version: "encryptionkey@0"
I20260808 07:01:42.362022 16270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:01:42.363067 16286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:42.363284 16270 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:42.363410 16270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "88e89354ac114e9b878e1be757114823"
format_stamp: "Formatted at 2026-08-08 07:01:42 on dist-test-slave-8tdl"
server_key: "f396646bdf3d18710fd22dfe45352011"
server_key_iv: "a8d06558509ca62cb0c76c8787f9cfd2"
server_key_version: "encryptionkey@0"
I20260808 07:01:42.363493 16270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:42.371467 16270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:42.373978 16270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:42.374187 16270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:42.374984 16270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:42.375051 16270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:42.375088 16270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:42.375110 16270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:01:42 dist-test-slave-8tdl krb5kdc[15866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172502, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:42.383368 16270 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:38209
I20260808 07:01:42.383562 16404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:38209 every 8 connection(s)
I20260808 07:01:42.383924 16270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:01:42.386729 15963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.375440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:36981 (local address 127.13.207.62:38167)
0808 07:01:42.375536 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:42.375539 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:42.375825 (+ 286us) server_negotiation.cc:407] Connection header received
0808 07:01:42.376611 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:42.376612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:42.376628 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:42.376711 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:42.377710 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.378295 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.379551 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.379754 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.379844 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.383139 (+ 3295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:42.383140 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:42.383141 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:42.383169 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:42.384872 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:42.385326 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:42.385327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:42.385327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:42.385396 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:42.385883 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:42.385884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:42.385884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:42.386040 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:42.386107 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:42.386487 (+ 380us) server_negotiation.cc:299] Negotiation successful
0808 07:01:42.386573 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 07:01:42.386938 16295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.375425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38167 (local address 127.13.207.3:36981)
0808 07:01:42.375653 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:42.375712 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:42.376455 (+ 743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:42.376799 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:42.376804 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:42.377252 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:42.377591 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.377599 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.378395 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.378397 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.379413 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.379416 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.379873 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.379875 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.380028 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.380666 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:42.380676 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:42.382882 (+ 2206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:42.384988 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:42.384994 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:42.384997 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:42.385241 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:42.385620 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:42.385621 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:42.385622 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:42.385775 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:42.386137 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:42.386142 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:42.386293 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 07:01:42.386404 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 07:01:42.386596 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":126,"threads_started":1}
I20260808 07:01:42.387611 16405 heartbeater.cc:347] Connected to a master server at 127.13.207.62:38167
I20260808 07:01:42.387727 16405 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:42.387924 16405 heartbeater.cc:511] Master 127.13.207.62:38167 requested a full tablet report, sending...
I20260808 07:01:42.388590 15912 ts_manager.cc:194] Registered new tserver with Master: 88e89354ac114e9b878e1be757114823 (127.13.207.3:38209)
I20260808 07:01:42.388899 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 16270
I20260808 07:01:42.388998 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:01:42.389145 15912 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:36981
I20260808 07:01:42.389176 14140 external_mini_cluster.cc:1428] Setting key d9bc4e41f517325b25f807d46f1f0a3b
I20260808 07:01:42.390013 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:01:42.396222 15963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.389067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:54809 (local address 127.13.207.62:38167)
0808 07:01:42.389111 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:42.389113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:42.389358 (+ 245us) server_negotiation.cc:407] Connection header received
0808 07:01:42.389463 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:42.389463 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:42.389474 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:42.389505 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:42.390065 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.390795 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.391539 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.391845 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.391924 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.393236 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:42.393238 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:42.393238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:42.393245 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:42.394885 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:42.395339 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:42.395340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:42.395341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:42.395402 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:42.395656 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:42.395657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:42.395657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:42.395785 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:42.395842 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:42.396020 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 07:01:42.396121 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:01:42.396277 16412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.389121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38167 (local address 127.13.207.3:54809)
0808 07:01:42.389293 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:42.389304 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:42.389387 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:42.389569 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:42.389570 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:42.389796 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:42.389864 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.389866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.390918 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.390919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.391447 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.391449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.391980 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.391981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.392040 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.392567 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:42.392573 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:42.393125 (+ 552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:42.394975 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:42.394976 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:42.394977 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:42.395238 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:42.395480 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:42.395482 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:42.395482 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:42.395554 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:42.395857 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:42.395858 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:42.395918 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 07:01:42.395994 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 07:01:42.396119 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":102,"threads_started":1}
Aug 08 07:01:42 dist-test-slave-8tdl krb5kdc[15866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:01:42 dist-test-slave-8tdl krb5kdc[15866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:42.417902 15963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.409659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40846 (local address 127.13.207.62:38167)
0808 07:01:42.409785 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:42.409787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:42.409800 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:01:42.409827 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:42.409828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:42.409839 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:42.410020 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:42.410610 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.411282 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.412156 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.412361 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.412428 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.414927 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:42.414929 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:42.414929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:42.414937 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:42.416451 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:42.417019 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:42.417020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:42.417021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:42.417085 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:42.417362 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:42.417363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:42.417364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:42.417505 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:42.417606 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:42.417754 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 07:01:42.417826 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260808 07:01:42.420349 15912 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40846:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:01:42.422618 15912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:01:42.430792 16428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.426512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41366 (local address 127.13.207.2:34543)
0808 07:01:42.426764 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:42.426768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:42.426777 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:01:42.426841 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:42.426843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:42.426861 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:42.426961 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:42.427063 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.428260 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.429775 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.430307 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.430400 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.430468 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:42.430600 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 07:01:42.430638 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":115,"threads_started":1}
I20260808 07:01:42.430792 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.426085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:34543 (local address 127.0.0.1:41366)
0808 07:01:42.426265 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:42.426311 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:42.426390 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:42.426961 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:42.426962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:42.426964 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:42.427003 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.427004 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.428379 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.428380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.429675 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.429677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.430400 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.430401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.430482 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.430498 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 07:01:42.430575 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 07:01:42.430637 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":52,"spinlock_wait_cycles":896,"thread_start_us":110,"threads_started":1}
I20260808 07:01:42.430935 16427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.426354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43620 (local address 127.13.207.3:38209)
0808 07:01:42.426732 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:42.426737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:42.426832 (+ 95us) server_negotiation.cc:407] Connection header received
0808 07:01:42.426897 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:42.426900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:42.426915 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:42.426993 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:42.427207 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.428299 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.429597 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.430365 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.430454 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.430515 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:42.430791 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 07:01:42.430837 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":97,"threads_started":1}
I20260808 07:01:42.431103 16426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.426294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38364 (local address 127.13.207.1:36077)
0808 07:01:42.426481 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:42.426487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:42.426571 (+ 84us) server_negotiation.cc:407] Connection header received
0808 07:01:42.426641 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:42.426643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:42.426659 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:42.427156 (+ 497us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:42.427219 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.427958 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.429644 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.430365 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.430841 (+ 476us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.430910 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:42.430949 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 07:01:42.430997 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":115,"threads_started":1}
I20260808 07:01:42.431275 16424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.426089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:38209 (local address 127.0.0.1:43620)
0808 07:01:42.426752 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:42.426764 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:42.426825 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:42.427062 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:42.427063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:42.427066 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:42.427123 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.427126 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.428381 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.428382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.429501 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.429504 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.430548 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.430548 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.430607 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.430611 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:01:42.430656 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 07:01:42.430726 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"mutex_wait_us":89,"thread_start_us":107,"threads_started":1}
I20260808 07:01:42.430811 16425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.426085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:36077 (local address 127.0.0.1:38364)
0808 07:01:42.426265 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:42.426517 (+ 252us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:42.426578 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:42.426848 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:42.426852 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:42.426855 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:42.426947 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.426950 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.428075 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.428076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.429561 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.429564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.430475 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.430476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.430560 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.430565 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:01:42.430597 (+ 32us) client_negotiation.cc:240] Negotiation successful
0808 07:01:42.430657 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":2,"thread_start_us":64,"threads_started":1}
I20260808 07:01:42.432113 16336 tablet_service.cc:1467] Processing CreateTablet for tablet aef53b649ddf4b7183f0c5d19efc958e (DEFAULT_TABLE table=test-table [id=ed98970f1c474f8d858d2cecf076b633]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:42.432092 16195 tablet_service.cc:1467] Processing CreateTablet for tablet aef53b649ddf4b7183f0c5d19efc958e (DEFAULT_TABLE table=test-table [id=ed98970f1c474f8d858d2cecf076b633]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:42.432251 16050 tablet_service.cc:1467] Processing CreateTablet for tablet aef53b649ddf4b7183f0c5d19efc958e (DEFAULT_TABLE table=test-table [id=ed98970f1c474f8d858d2cecf076b633]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:42.432477 16195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aef53b649ddf4b7183f0c5d19efc958e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:42.432487 16336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aef53b649ddf4b7183f0c5d19efc958e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:42.432555 16050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aef53b649ddf4b7183f0c5d19efc958e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:42.435627 16429 tablet_bootstrap.cc:492] T aef53b649ddf4b7183f0c5d19efc958e P 88e89354ac114e9b878e1be757114823: Bootstrap starting.
I20260808 07:01:42.436221 16430 tablet_bootstrap.cc:492] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f: Bootstrap starting.
I20260808 07:01:42.436424 16431 tablet_bootstrap.cc:492] T aef53b649ddf4b7183f0c5d19efc958e P dd53f8d8a09a47d5b2ddcc7e05f098be: Bootstrap starting.
I20260808 07:01:42.436607 16429 tablet_bootstrap.cc:654] T aef53b649ddf4b7183f0c5d19efc958e P 88e89354ac114e9b878e1be757114823: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:42.436853 16430 tablet_bootstrap.cc:654] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:42.436952 16429 log.cc:824] T aef53b649ddf4b7183f0c5d19efc958e P 88e89354ac114e9b878e1be757114823: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:42.437043 16431 tablet_bootstrap.cc:654] T aef53b649ddf4b7183f0c5d19efc958e P dd53f8d8a09a47d5b2ddcc7e05f098be: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:42.437207 16430 log.cc:824] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:42.437389 16431 log.cc:824] T aef53b649ddf4b7183f0c5d19efc958e P dd53f8d8a09a47d5b2ddcc7e05f098be: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:42.437813 16429 tablet_bootstrap.cc:492] T aef53b649ddf4b7183f0c5d19efc958e P 88e89354ac114e9b878e1be757114823: No bootstrap required, opened a new log
I20260808 07:01:42.437904 16429 ts_tablet_manager.cc:1397] T aef53b649ddf4b7183f0c5d19efc958e P 88e89354ac114e9b878e1be757114823: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:01:42.438136 16431 tablet_bootstrap.cc:492] T aef53b649ddf4b7183f0c5d19efc958e P dd53f8d8a09a47d5b2ddcc7e05f098be: No bootstrap required, opened a new log
I20260808 07:01:42.438233 16431 ts_tablet_manager.cc:1397] T aef53b649ddf4b7183f0c5d19efc958e P dd53f8d8a09a47d5b2ddcc7e05f098be: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:01:42.440276 16429 raft_consensus.cc:348] T aef53b649ddf4b7183f0c5d19efc958e P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.440495 16429 raft_consensus.cc:374] T aef53b649ddf4b7183f0c5d19efc958e P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:42.440542 16429 raft_consensus.cc:729] T aef53b649ddf4b7183f0c5d19efc958e P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88e89354ac114e9b878e1be757114823, State: Initialized, Role: FOLLOWER
I20260808 07:01:42.440472 16431 raft_consensus.cc:348] T aef53b649ddf4b7183f0c5d19efc958e P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.440667 16431 raft_consensus.cc:374] T aef53b649ddf4b7183f0c5d19efc958e P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:42.440709 16431 raft_consensus.cc:729] T aef53b649ddf4b7183f0c5d19efc958e P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd53f8d8a09a47d5b2ddcc7e05f098be, State: Initialized, Role: FOLLOWER
I20260808 07:01:42.440704 16429 consensus_queue.cc:260] T aef53b649ddf4b7183f0c5d19efc958e P 88e89354ac114e9b878e1be757114823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.440757 16430 tablet_bootstrap.cc:492] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f: No bootstrap required, opened a new log
I20260808 07:01:42.440833 16430 ts_tablet_manager.cc:1397] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 07:01:42.440862 16431 consensus_queue.cc:260] T aef53b649ddf4b7183f0c5d19efc958e P dd53f8d8a09a47d5b2ddcc7e05f098be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.441033 16429 ts_tablet_manager.cc:1428] T aef53b649ddf4b7183f0c5d19efc958e P 88e89354ac114e9b878e1be757114823: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:01:42.441154 16431 ts_tablet_manager.cc:1428] T aef53b649ddf4b7183f0c5d19efc958e P dd53f8d8a09a47d5b2ddcc7e05f098be: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:01:42.441179 16405 heartbeater.cc:503] Master 127.13.207.62:38167 was elected leader, sending a full tablet report...
I20260808 07:01:42.441284 16261 heartbeater.cc:503] Master 127.13.207.62:38167 was elected leader, sending a full tablet report...
I20260808 07:01:42.442978 16430 raft_consensus.cc:348] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.443122 16430 raft_consensus.cc:374] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:42.443145 16430 raft_consensus.cc:729] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5568508efd67416d8c5a791cef3fd44f, State: Initialized, Role: FOLLOWER
I20260808 07:01:42.443269 16430 consensus_queue.cc:260] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.443575 16430 ts_tablet_manager.cc:1428] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:01:42.443656 16119 heartbeater.cc:503] Master 127.13.207.62:38167 was elected leader, sending a full tablet report...
W20260808 07:01:42.462778 16262 tablet.cc:2369] T aef53b649ddf4b7183f0c5d19efc958e P dd53f8d8a09a47d5b2ddcc7e05f098be: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:01:42.530196 16120 tablet.cc:2369] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:01:42.634786 16406 tablet.cc:2369] T aef53b649ddf4b7183f0c5d19efc958e P 88e89354ac114e9b878e1be757114823: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:01:42.652331 16437 raft_consensus.cc:482] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:01:42.652532 16437 raft_consensus.cc:504] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.653002 16437 leader_election.cc:288] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd53f8d8a09a47d5b2ddcc7e05f098be (127.13.207.2:34543), 88e89354ac114e9b878e1be757114823 (127.13.207.3:38209)
I20260808 07:01:42.656663 16439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.653240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:38209 (local address 127.13.207.1:46437)
0808 07:01:42.653385 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:42.653396 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:42.653461 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:42.653624 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:42.653624 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:42.653627 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:42.653716 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.653718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.654405 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.654406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.655650 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.655653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.656320 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.656322 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.656406 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.656411 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:01:42.656492 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 07:01:42.656542 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":2,"thread_start_us":82,"threads_started":1}
I20260808 07:01:42.656683 16427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.653407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:46437 (local address 127.13.207.3:38209)
0808 07:01:42.653494 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:42.653498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:42.653512 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:01:42.653539 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:42.653540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:42.653548 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:42.653603 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:42.653775 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.654319 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.655748 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.656216 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.656278 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.656285 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:42.656526 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 07:01:42.656571 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 07:01:42.656984 16438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.653249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:34543 (local address 127.13.207.1:50321)
0808 07:01:42.653651 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:42.653660 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:42.653705 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:42.653852 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:42.653853 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:42.653855 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:42.653902 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.653903 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.654777 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.654778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.655888 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.655891 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.656665 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.656666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.656746 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.656751 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:01:42.656861 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 07:01:42.656904 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":13,"thread_start_us":70,"threads_started":1}
I20260808 07:01:42.657068 16428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.653328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:50321 (local address 127.13.207.2:34543)
0808 07:01:42.653379 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:42.653382 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:42.653728 (+ 346us) server_negotiation.cc:407] Connection header received
0808 07:01:42.653761 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:42.653762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:42.653773 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:42.653839 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:42.654046 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.654648 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.656004 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.656554 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.656629 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.656638 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:42.656878 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 07:01:42.656925 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:01:42.657356 16215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aef53b649ddf4b7183f0c5d19efc958e" candidate_uuid: "5568508efd67416d8c5a791cef3fd44f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" is_pre_election: true
I20260808 07:01:42.657425 16356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aef53b649ddf4b7183f0c5d19efc958e" candidate_uuid: "5568508efd67416d8c5a791cef3fd44f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88e89354ac114e9b878e1be757114823" is_pre_election: true
I20260808 07:01:42.657536 16215 raft_consensus.cc:2454] T aef53b649ddf4b7183f0c5d19efc958e P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5568508efd67416d8c5a791cef3fd44f in term 0.
I20260808 07:01:42.657586 16356 raft_consensus.cc:2454] T aef53b649ddf4b7183f0c5d19efc958e P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5568508efd67416d8c5a791cef3fd44f in term 0.
I20260808 07:01:42.657938 16002 leader_election.cc:302] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5568508efd67416d8c5a791cef3fd44f, 88e89354ac114e9b878e1be757114823, dd53f8d8a09a47d5b2ddcc7e05f098be; no voters:
I20260808 07:01:42.658102 16437 raft_consensus.cc:2781] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:01:42.658164 16437 raft_consensus.cc:482] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:01:42.658205 16437 raft_consensus.cc:3037] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:42.659085 16437 raft_consensus.cc:504] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.659258 16437 leader_election.cc:288] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f [CANDIDATE]: Term 1 election: Requested vote from peers dd53f8d8a09a47d5b2ddcc7e05f098be (127.13.207.2:34543), 88e89354ac114e9b878e1be757114823 (127.13.207.3:38209)
I20260808 07:01:42.659430 16215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aef53b649ddf4b7183f0c5d19efc958e" candidate_uuid: "5568508efd67416d8c5a791cef3fd44f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be"
I20260808 07:01:42.659493 16356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aef53b649ddf4b7183f0c5d19efc958e" candidate_uuid: "5568508efd67416d8c5a791cef3fd44f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88e89354ac114e9b878e1be757114823"
I20260808 07:01:42.659541 16215 raft_consensus.cc:3037] T aef53b649ddf4b7183f0c5d19efc958e P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:42.659608 16356 raft_consensus.cc:3037] T aef53b649ddf4b7183f0c5d19efc958e P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:42.660313 16215 raft_consensus.cc:2454] T aef53b649ddf4b7183f0c5d19efc958e P dd53f8d8a09a47d5b2ddcc7e05f098be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5568508efd67416d8c5a791cef3fd44f in term 1.
I20260808 07:01:42.660485 16356 raft_consensus.cc:2454] T aef53b649ddf4b7183f0c5d19efc958e P 88e89354ac114e9b878e1be757114823 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5568508efd67416d8c5a791cef3fd44f in term 1.
I20260808 07:01:42.660583 16002 leader_election.cc:302] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5568508efd67416d8c5a791cef3fd44f, dd53f8d8a09a47d5b2ddcc7e05f098be; no voters:
I20260808 07:01:42.660723 16437 raft_consensus.cc:2781] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:42.660877 16437 raft_consensus.cc:686] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f [term 1 LEADER]: Becoming Leader. State: Replica: 5568508efd67416d8c5a791cef3fd44f, State: Running, Role: LEADER
I20260808 07:01:42.660990 16437 consensus_queue.cc:237] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.661978 15912 catalog_manager.cc:5495] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5568508efd67416d8c5a791cef3fd44f (127.13.207.1). New cstate: current_term: 1 leader_uuid: "5568508efd67416d8c5a791cef3fd44f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } health_report { overall_health: UNKNOWN } } }
Aug 08 07:01:42 dist-test-slave-8tdl krb5kdc[15866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172501, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:42.752235 15963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.743835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40852 (local address 127.13.207.62:38167)
0808 07:01:42.743956 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:42.743959 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:42.744113 (+ 154us) server_negotiation.cc:407] Connection header received
0808 07:01:42.744210 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:42.744210 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:42.744222 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:42.744281 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:42.744795 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.745295 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.746076 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.746271 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.746364 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.749034 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:42.749037 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:42.749037 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:42.749045 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:42.750834 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:42.751307 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:42.751309 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:42.751309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:42.751379 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:42.751614 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:42.751615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:42.751616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:42.751752 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:42.751836 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:42.752010 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 07:01:42.752108 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 07:01:42.752336 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.743735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38167 (local address 127.0.0.1:40852)
0808 07:01:42.744021 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:42.744031 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:42.744134 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:42.744309 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:42.744310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:42.744655 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:42.744703 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.744706 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.745397 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.745398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.745982 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.745984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.746360 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.746361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.746411 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.747151 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:42.747156 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:42.748921 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:42.750957 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:42.750970 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:42.750979 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:42.751202 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:42.751472 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:42.751473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:42.751473 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:42.751543 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:42.751825 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:42.751835 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:42.751894 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 07:01:42.751981 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 07:01:42.752175 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":70,"thread_start_us":124,"threads_started":1}
I20260808 07:01:42.754823 15912 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:40852:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 07:01:42.755038 15912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:01:42.757146 16336 tablet_service.cc:1467] Processing CreateTablet for tablet 51364d7c1add47ecbb623353956a9fbb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bd6ace8694eb4f6a983329f87efba46c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:01:42.757155 16195 tablet_service.cc:1467] Processing CreateTablet for tablet 51364d7c1add47ecbb623353956a9fbb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bd6ace8694eb4f6a983329f87efba46c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:01:42.757265 16050 tablet_service.cc:1467] Processing CreateTablet for tablet 51364d7c1add47ecbb623353956a9fbb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bd6ace8694eb4f6a983329f87efba46c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:01:42.757360 16336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51364d7c1add47ecbb623353956a9fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:42.757361 16195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51364d7c1add47ecbb623353956a9fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:42.757444 16050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51364d7c1add47ecbb623353956a9fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:42.758960 16429 tablet_bootstrap.cc:492] T 51364d7c1add47ecbb623353956a9fbb P 88e89354ac114e9b878e1be757114823: Bootstrap starting.
I20260808 07:01:42.759141 16430 tablet_bootstrap.cc:492] T 51364d7c1add47ecbb623353956a9fbb P 5568508efd67416d8c5a791cef3fd44f: Bootstrap starting.
I20260808 07:01:42.759253 16431 tablet_bootstrap.cc:492] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be: Bootstrap starting.
I20260808 07:01:42.759606 16431 tablet_bootstrap.cc:654] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:42.759660 16430 tablet_bootstrap.cc:654] T 51364d7c1add47ecbb623353956a9fbb P 5568508efd67416d8c5a791cef3fd44f: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:42.759660 16429 tablet_bootstrap.cc:654] T 51364d7c1add47ecbb623353956a9fbb P 88e89354ac114e9b878e1be757114823: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:42.760289 16431 tablet_bootstrap.cc:492] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be: No bootstrap required, opened a new log
I20260808 07:01:42.760367 16431 ts_tablet_manager.cc:1397] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:42.760555 16431 raft_consensus.cc:348] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.760648 16431 raft_consensus.cc:374] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:42.760677 16431 raft_consensus.cc:729] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd53f8d8a09a47d5b2ddcc7e05f098be, State: Initialized, Role: FOLLOWER
I20260808 07:01:42.760720 16430 tablet_bootstrap.cc:492] T 51364d7c1add47ecbb623353956a9fbb P 5568508efd67416d8c5a791cef3fd44f: No bootstrap required, opened a new log
I20260808 07:01:42.760738 16431 consensus_queue.cc:260] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.760782 16430 ts_tablet_manager.cc:1397] T 51364d7c1add47ecbb623353956a9fbb P 5568508efd67416d8c5a791cef3fd44f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:01:42.760908 16431 ts_tablet_manager.cc:1428] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:42.760959 16430 raft_consensus.cc:348] T 51364d7c1add47ecbb623353956a9fbb P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.761037 16430 raft_consensus.cc:374] T 51364d7c1add47ecbb623353956a9fbb P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:42.760979 16436 tablet_replica.cc:439] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } } }
I20260808 07:01:42.761061 16430 raft_consensus.cc:729] T 51364d7c1add47ecbb623353956a9fbb P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5568508efd67416d8c5a791cef3fd44f, State: Initialized, Role: FOLLOWER
I20260808 07:01:42.761086 16436 tablet_replica.cc:442] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:01:42.761112 16430 consensus_queue.cc:260] T 51364d7c1add47ecbb623353956a9fbb P 5568508efd67416d8c5a791cef3fd44f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.761217 16430 ts_tablet_manager.cc:1428] T 51364d7c1add47ecbb623353956a9fbb P 5568508efd67416d8c5a791cef3fd44f: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:01:42.761366 16437 tablet_replica.cc:439] T 51364d7c1add47ecbb623353956a9fbb P 5568508efd67416d8c5a791cef3fd44f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } } }
I20260808 07:01:42.761471 16429 tablet_bootstrap.cc:492] T 51364d7c1add47ecbb623353956a9fbb P 88e89354ac114e9b878e1be757114823: No bootstrap required, opened a new log
I20260808 07:01:42.761464 16437 tablet_replica.cc:442] T 51364d7c1add47ecbb623353956a9fbb P 5568508efd67416d8c5a791cef3fd44f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:01:42.761518 16429 ts_tablet_manager.cc:1397] T 51364d7c1add47ecbb623353956a9fbb P 88e89354ac114e9b878e1be757114823: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 07:01:42.761687 16429 raft_consensus.cc:348] T 51364d7c1add47ecbb623353956a9fbb P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.761785 16429 raft_consensus.cc:374] T 51364d7c1add47ecbb623353956a9fbb P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:42.761808 16429 raft_consensus.cc:729] T 51364d7c1add47ecbb623353956a9fbb P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88e89354ac114e9b878e1be757114823, State: Initialized, Role: FOLLOWER
I20260808 07:01:42.761868 16429 consensus_queue.cc:260] T 51364d7c1add47ecbb623353956a9fbb P 88e89354ac114e9b878e1be757114823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.761991 16429 ts_tablet_manager.cc:1428] T 51364d7c1add47ecbb623353956a9fbb P 88e89354ac114e9b878e1be757114823: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:01:42.762094 16435 tablet_replica.cc:439] T 51364d7c1add47ecbb623353956a9fbb P 88e89354ac114e9b878e1be757114823: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } } }
I20260808 07:01:42.762204 16435 tablet_replica.cc:442] T 51364d7c1add47ecbb623353956a9fbb P 88e89354ac114e9b878e1be757114823: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:01:42.766217 16436 raft_consensus.cc:482] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:01:42.766315 16436 raft_consensus.cc:504] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.766752 16436 leader_election.cc:288] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5568508efd67416d8c5a791cef3fd44f (127.13.207.1:36077), 88e89354ac114e9b878e1be757114823 (127.13.207.3:38209)
I20260808 07:01:42.770335 16427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.767118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:60215 (local address 127.13.207.3:38209)
0808 07:01:42.767170 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:42.767172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:42.767271 (+ 99us) server_negotiation.cc:407] Connection header received
0808 07:01:42.767351 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:42.767352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:42.767361 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:42.767419 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:42.767585 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.768077 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.769542 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.769919 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.770019 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.770030 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:42.770210 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 07:01:42.770256 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:01:42.770344 16454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.766942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:38209 (local address 127.13.207.2:60215)
0808 07:01:42.767184 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:42.767194 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:42.767290 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:42.767423 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:42.767424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:42.767426 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:42.767503 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.767505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.768169 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.768170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.769438 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.769442 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.770045 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.770046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.770121 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.770125 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:01:42.770190 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 07:01:42.770241 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":74,"threads_started":1}
I20260808 07:01:42.770423 16426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.767072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:59777 (local address 127.13.207.1:36077)
0808 07:01:42.767127 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:42.767130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:42.767352 (+ 222us) server_negotiation.cc:407] Connection header received
0808 07:01:42.767459 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:42.767460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:42.767471 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:42.767542 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:42.767730 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.768268 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.769584 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.769992 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.770083 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.770091 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:42.770286 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 07:01:42.770336 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:01:42.770495 16453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.766991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:36077 (local address 127.13.207.2:59777)
0808 07:01:42.767275 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:42.767287 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:42.767384 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:42.767562 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:42.767563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:42.767565 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:42.767648 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.767650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.768358 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.768359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.769469 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.769472 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.770082 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.770083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.770183 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.770187 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:01:42.770275 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 07:01:42.770327 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":85,"threads_started":1}
I20260808 07:01:42.770743 16356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51364d7c1add47ecbb623353956a9fbb" candidate_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88e89354ac114e9b878e1be757114823" is_pre_election: true
I20260808 07:01:42.770888 16356 raft_consensus.cc:2454] T 51364d7c1add47ecbb623353956a9fbb P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd53f8d8a09a47d5b2ddcc7e05f098be in term 0.
I20260808 07:01:42.770875 16073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51364d7c1add47ecbb623353956a9fbb" candidate_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5568508efd67416d8c5a791cef3fd44f" is_pre_election: true
I20260808 07:01:42.771121 16073 raft_consensus.cc:2454] T 51364d7c1add47ecbb623353956a9fbb P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd53f8d8a09a47d5b2ddcc7e05f098be in term 0.
I20260808 07:01:42.771098 16144 leader_election.cc:302] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88e89354ac114e9b878e1be757114823, dd53f8d8a09a47d5b2ddcc7e05f098be; no voters:
I20260808 07:01:42.771297 16436 raft_consensus.cc:2781] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:01:42.771364 16436 raft_consensus.cc:482] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:01:42.771389 16436 raft_consensus.cc:3037] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:42.772302 16436 raft_consensus.cc:504] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.772526 16436 leader_election.cc:288] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be [CANDIDATE]: Term 1 election: Requested vote from peers 5568508efd67416d8c5a791cef3fd44f (127.13.207.1:36077), 88e89354ac114e9b878e1be757114823 (127.13.207.3:38209)
I20260808 07:01:42.772720 16073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51364d7c1add47ecbb623353956a9fbb" candidate_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5568508efd67416d8c5a791cef3fd44f"
I20260808 07:01:42.772742 16356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51364d7c1add47ecbb623353956a9fbb" candidate_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88e89354ac114e9b878e1be757114823"
I20260808 07:01:42.772814 16073 raft_consensus.cc:3037] T 51364d7c1add47ecbb623353956a9fbb P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:42.772838 16356 raft_consensus.cc:3037] T 51364d7c1add47ecbb623353956a9fbb P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:42.773546 16356 raft_consensus.cc:2454] T 51364d7c1add47ecbb623353956a9fbb P 88e89354ac114e9b878e1be757114823 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd53f8d8a09a47d5b2ddcc7e05f098be in term 1.
I20260808 07:01:42.773553 16073 raft_consensus.cc:2454] T 51364d7c1add47ecbb623353956a9fbb P 5568508efd67416d8c5a791cef3fd44f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd53f8d8a09a47d5b2ddcc7e05f098be in term 1.
I20260808 07:01:42.773777 16144 leader_election.cc:302] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88e89354ac114e9b878e1be757114823, dd53f8d8a09a47d5b2ddcc7e05f098be; no voters:
I20260808 07:01:42.773893 16436 raft_consensus.cc:2781] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:42.774024 16436 raft_consensus.cc:686] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be [term 1 LEADER]: Becoming Leader. State: Replica: dd53f8d8a09a47d5b2ddcc7e05f098be, State: Running, Role: LEADER
I20260808 07:01:42.774154 16436 consensus_queue.cc:237] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.774916 16455 tablet_replica.cc:439] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be: TxnStatusTablet state changed. Reason: New leader dd53f8d8a09a47d5b2ddcc7e05f098be. Latest consensus state: current_term: 1 leader_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } } }
I20260808 07:01:42.775170 16455 tablet_replica.cc:442] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:01:42.775488 16457 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 07:01:42.775616 16457 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 07:01:42.775789 15912 catalog_manager.cc:5495] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be reported cstate change: term changed from 0 to 1, leader changed from <none> to dd53f8d8a09a47d5b2ddcc7e05f098be (127.13.207.2). New cstate: current_term: 1 leader_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } health_report { overall_health: UNKNOWN } } }
I20260808 07:01:42.782181 16428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.778869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41378 (local address 127.13.207.2:34543)
0808 07:01:42.778919 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:42.778921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:42.779002 (+ 81us) server_negotiation.cc:407] Connection header received
0808 07:01:42.779253 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:42.779254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:42.779276 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:42.779348 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:01:42.779584 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.780086 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.781232 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.781495 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.781567 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.781818 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:01:42.781929 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:01:42.782021 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:42.782091 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 07:01:42.782116 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:01:42.782236 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.778836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:34543 (local address 127.0.0.1:41378)
0808 07:01:42.778925 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:42.778937 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:42.779025 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:42.779379 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:42.779380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:42.779382 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:01:42.779437 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.779440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.780176 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.780177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.781111 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.781114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.781591 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.781592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.781668 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.781692 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:01:42.782015 (+ 323us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:01:42.782025 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 07:01:42.782086 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 07:01:42.782147 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 07:01:42.784550 16073 raft_consensus.cc:1261] T 51364d7c1add47ecbb623353956a9fbb P 5568508efd67416d8c5a791cef3fd44f [term 1 FOLLOWER]: Refusing update from remote peer dd53f8d8a09a47d5b2ddcc7e05f098be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:01:42.784603 16356 raft_consensus.cc:1261] T 51364d7c1add47ecbb623353956a9fbb P 88e89354ac114e9b878e1be757114823 [term 1 FOLLOWER]: Refusing update from remote peer dd53f8d8a09a47d5b2ddcc7e05f098be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:01:42.785013 16436 consensus_queue.cc:1035] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be [LEADER]: Connected to new peer: Peer: permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:01:42.785192 16455 consensus_queue.cc:1035] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be [LEADER]: Connected to new peer: Peer: permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:01:42.786533 16437 tablet_replica.cc:439] T 51364d7c1add47ecbb623353956a9fbb P 5568508efd67416d8c5a791cef3fd44f: TxnStatusTablet state changed. Reason: New leader dd53f8d8a09a47d5b2ddcc7e05f098be. Latest consensus state: current_term: 1 leader_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } } }
I20260808 07:01:42.786665 16437 tablet_replica.cc:442] T 51364d7c1add47ecbb623353956a9fbb P 5568508efd67416d8c5a791cef3fd44f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:01:42.786770 16435 tablet_replica.cc:439] T 51364d7c1add47ecbb623353956a9fbb P 88e89354ac114e9b878e1be757114823: TxnStatusTablet state changed. Reason: New leader dd53f8d8a09a47d5b2ddcc7e05f098be. Latest consensus state: current_term: 1 leader_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } } }
I20260808 07:01:42.786900 16435 tablet_replica.cc:442] T 51364d7c1add47ecbb623353956a9fbb P 88e89354ac114e9b878e1be757114823: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:01:42.788728 16455 tablet_replica.cc:439] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } } }
I20260808 07:01:42.788849 16436 tablet_replica.cc:439] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } } }
I20260808 07:01:42.788977 16436 tablet_replica.cc:442] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:01:42.789253 16435 tablet_replica.cc:439] T 51364d7c1add47ecbb623353956a9fbb P 88e89354ac114e9b878e1be757114823: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } } }
I20260808 07:01:42.789372 16435 tablet_replica.cc:442] T 51364d7c1add47ecbb623353956a9fbb P 88e89354ac114e9b878e1be757114823: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:01:42.789422 16437 tablet_replica.cc:439] T 51364d7c1add47ecbb623353956a9fbb P 5568508efd67416d8c5a791cef3fd44f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } } }
I20260808 07:01:42.789510 16437 tablet_replica.cc:442] T 51364d7c1add47ecbb623353956a9fbb P 5568508efd67416d8c5a791cef3fd44f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:01:42.788859 16455 tablet_replica.cc:442] T 51364d7c1add47ecbb623353956a9fbb P dd53f8d8a09a47d5b2ddcc7e05f098be: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:01:42.789824 16461 mvcc.cc:205] Tried to move back new op lower bound from 7316162571402399744 to 7316162571363586048. Current Snapshot: MvccSnapshot[applied={T|T < 7316162571402399744}]
I20260808 07:01:42.789922 16463 mvcc.cc:205] Tried to move back new op lower bound from 7316162571402399744 to 7316162571363586048. Current Snapshot: MvccSnapshot[applied={T|T < 7316162571402399744}]
I20260808 07:01:42.790452 16462 mvcc.cc:205] Tried to move back new op lower bound from 7316162571402399744 to 7316162571363586048. Current Snapshot: MvccSnapshot[applied={T|T < 7316162571402399744}]
I20260808 07:01:42.795360 16426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.792380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38368 (local address 127.13.207.1:36077)
0808 07:01:42.792426 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:42.792428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:42.792437 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:01:42.792501 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:42.792502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:42.792521 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:42.792606 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:01:42.792701 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.793476 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.794327 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.794643 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.794754 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.794946 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:01:42.795082 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:01:42.795178 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:42.795255 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 07:01:42.795277 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:01:42.801131 16428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.798140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:55565 (local address 127.13.207.2:34543)
0808 07:01:42.798193 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:42.798195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:42.798353 (+ 158us) server_negotiation.cc:407] Connection header received
0808 07:01:42.798411 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:42.798412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:42.798423 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:42.798463 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:01:42.798657 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.799213 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.800420 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.800604 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.800661 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.800893 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:01:42.800921 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:01:42.800965 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:42.801052 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 07:01:42.801069 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 07:01:42.801203 16473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.798044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:34543 (local address 127.13.207.1:55565)
0808 07:01:42.798263 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:42.798276 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:42.798348 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:42.798502 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:42.798503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:42.798505 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:01:42.798575 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.798577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.799301 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.799303 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.800311 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.800314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.800668 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.800669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.800746 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.800767 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:01:42.800994 (+ 227us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:01:42.800996 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:01:42.801048 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:01:42.801094 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":116,"threads_started":1}
I20260808 07:01:42.804557 16356 raft_consensus.cc:1261] T aef53b649ddf4b7183f0c5d19efc958e P 88e89354ac114e9b878e1be757114823 [term 1 FOLLOWER]: Refusing update from remote peer 5568508efd67416d8c5a791cef3fd44f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:01:42.804693 16215 raft_consensus.cc:1261] T aef53b649ddf4b7183f0c5d19efc958e P dd53f8d8a09a47d5b2ddcc7e05f098be [term 1 FOLLOWER]: Refusing update from remote peer 5568508efd67416d8c5a791cef3fd44f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:01:42.804903 16440 consensus_queue.cc:1035] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f [LEADER]: Connected to new peer: Peer: permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:01:42.805157 16437 consensus_queue.cc:1035] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:01:42.815363 16426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.812321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:46481 (local address 127.13.207.1:36077)
0808 07:01:42.812361 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:42.812363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:42.812372 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:01:42.812458 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:42.812459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:42.812469 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:42.812527 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:01:42.812714 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.813322 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.814538 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.814858 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.814945 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.815124 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:01:42.815163 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:01:42.815236 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:42.815253 (+ 17us) server_negotiation.cc:299] Negotiation successful
0808 07:01:42.815278 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:01:42.815519 16479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:42.812039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:36077 (local address 127.13.207.2:46481)
0808 07:01:42.812285 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:42.812299 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:42.812366 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:42.812531 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:42.812532 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:42.812533 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:01:42.812638 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.812639 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.813463 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.813464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.814420 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:42.814432 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:42.814965 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:42.814968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:42.815022 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:42.815039 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:01:42.815224 (+ 185us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:01:42.815225 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:01:42.815250 (+ 25us) client_negotiation.cc:240] Negotiation successful
0808 07:01:42.815287 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":176,"threads_started":1}
I20260808 07:01:42.825656 15912 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:01:42.825803 15912 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:01:42.826835 15912 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830: Sending DeleteTablet for 3 replicas of tablet aef53b649ddf4b7183f0c5d19efc958e
I20260808 07:01:42.827241 16336 tablet_service.cc:1514] Processing DeleteTablet for tablet aef53b649ddf4b7183f0c5d19efc958e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:42 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:43620
I20260808 07:01:42.827313 16195 tablet_service.cc:1514] Processing DeleteTablet for tablet aef53b649ddf4b7183f0c5d19efc958e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:42 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:41366
I20260808 07:01:42.827517 16050 tablet_service.cc:1514] Processing DeleteTablet for tablet aef53b649ddf4b7183f0c5d19efc958e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:42 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:38364
I20260808 07:01:42.827646 16485 tablet_replica.cc:330] T aef53b649ddf4b7183f0c5d19efc958e P dd53f8d8a09a47d5b2ddcc7e05f098be: stopping tablet replica
I20260808 07:01:42.827819 16485 raft_consensus.cc:2229] T aef53b649ddf4b7183f0c5d19efc958e P dd53f8d8a09a47d5b2ddcc7e05f098be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:01:42.827963 16485 raft_consensus.cc:2258] T aef53b649ddf4b7183f0c5d19efc958e P dd53f8d8a09a47d5b2ddcc7e05f098be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:01:42.828011 15912 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40846:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:01:42.828197 15912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:01:42.828842 16485 ts_tablet_manager.cc:1905] T aef53b649ddf4b7183f0c5d19efc958e P dd53f8d8a09a47d5b2ddcc7e05f098be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:01:42.829689 16050 tablet_service.cc:1467] Processing CreateTablet for tablet 8774bb94f82e4f148ea99a2c3b353514 (DEFAULT_TABLE table=test-table [id=34ec8f48ea6544eeaa71e31868523d91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:42.829726 16195 tablet_service.cc:1467] Processing CreateTablet for tablet 8774bb94f82e4f148ea99a2c3b353514 (DEFAULT_TABLE table=test-table [id=34ec8f48ea6544eeaa71e31868523d91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:42.829833 16050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8774bb94f82e4f148ea99a2c3b353514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:42.829855 16195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8774bb94f82e4f148ea99a2c3b353514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:42.831295 16430 tablet_bootstrap.cc:492] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f: Bootstrap starting.
I20260808 07:01:42.831439 16431 tablet_bootstrap.cc:492] T 8774bb94f82e4f148ea99a2c3b353514 P dd53f8d8a09a47d5b2ddcc7e05f098be: Bootstrap starting.
I20260808 07:01:42.831758 16430 tablet_bootstrap.cc:654] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:42.831874 16431 tablet_bootstrap.cc:654] T 8774bb94f82e4f148ea99a2c3b353514 P dd53f8d8a09a47d5b2ddcc7e05f098be: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:42.832163 16485 ts_tablet_manager.cc:1918] T aef53b649ddf4b7183f0c5d19efc958e P dd53f8d8a09a47d5b2ddcc7e05f098be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:01:42.832320 16485 log.cc:1196] T aef53b649ddf4b7183f0c5d19efc958e P dd53f8d8a09a47d5b2ddcc7e05f098be: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-1/wal/wals/aef53b649ddf4b7183f0c5d19efc958e
I20260808 07:01:42.832451 16336 tablet_service.cc:1467] Processing CreateTablet for tablet 8774bb94f82e4f148ea99a2c3b353514 (DEFAULT_TABLE table=test-table [id=34ec8f48ea6544eeaa71e31868523d91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:42.832607 16336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8774bb94f82e4f148ea99a2c3b353514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:42.832854 16485 ts_tablet_manager.cc:1939] T aef53b649ddf4b7183f0c5d19efc958e P dd53f8d8a09a47d5b2ddcc7e05f098be: Deleting consensus metadata
I20260808 07:01:42.833475 15898 catalog_manager.cc:4841] TS dd53f8d8a09a47d5b2ddcc7e05f098be (127.13.207.2:34543): tablet aef53b649ddf4b7183f0c5d19efc958e (table test-table [id=ed98970f1c474f8d858d2cecf076b633]) successfully deleted
I20260808 07:01:42.833981 16429 tablet_bootstrap.cc:492] T 8774bb94f82e4f148ea99a2c3b353514 P 88e89354ac114e9b878e1be757114823: Bootstrap starting.
I20260808 07:01:42.834161 16431 tablet_bootstrap.cc:492] T 8774bb94f82e4f148ea99a2c3b353514 P dd53f8d8a09a47d5b2ddcc7e05f098be: No bootstrap required, opened a new log
I20260808 07:01:42.834209 16431 ts_tablet_manager.cc:1397] T 8774bb94f82e4f148ea99a2c3b353514 P dd53f8d8a09a47d5b2ddcc7e05f098be: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 07:01:42.834380 16429 tablet_bootstrap.cc:654] T 8774bb94f82e4f148ea99a2c3b353514 P 88e89354ac114e9b878e1be757114823: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:42.834404 16431 raft_consensus.cc:348] T 8774bb94f82e4f148ea99a2c3b353514 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.834493 16431 raft_consensus.cc:374] T 8774bb94f82e4f148ea99a2c3b353514 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:42.834517 16431 raft_consensus.cc:729] T 8774bb94f82e4f148ea99a2c3b353514 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd53f8d8a09a47d5b2ddcc7e05f098be, State: Initialized, Role: FOLLOWER
I20260808 07:01:42.834566 16431 consensus_queue.cc:260] T 8774bb94f82e4f148ea99a2c3b353514 P dd53f8d8a09a47d5b2ddcc7e05f098be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.834692 16431 ts_tablet_manager.cc:1428] T 8774bb94f82e4f148ea99a2c3b353514 P dd53f8d8a09a47d5b2ddcc7e05f098be: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:42.835096 16430 tablet_bootstrap.cc:492] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f: No bootstrap required, opened a new log
I20260808 07:01:42.835163 16430 ts_tablet_manager.cc:1397] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260808 07:01:42.835289 16483 tablet_replica.cc:330] T aef53b649ddf4b7183f0c5d19efc958e P 88e89354ac114e9b878e1be757114823: stopping tablet replica
I20260808 07:01:42.835341 16430 raft_consensus.cc:348] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.835420 16430 raft_consensus.cc:374] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:42.835419 16483 raft_consensus.cc:2229] T aef53b649ddf4b7183f0c5d19efc958e P 88e89354ac114e9b878e1be757114823 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:01:42.835449 16430 raft_consensus.cc:729] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5568508efd67416d8c5a791cef3fd44f, State: Initialized, Role: FOLLOWER
I20260808 07:01:42.835494 16483 raft_consensus.cc:2258] T aef53b649ddf4b7183f0c5d19efc958e P 88e89354ac114e9b878e1be757114823 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:01:42.835492 16430 consensus_queue.cc:260] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.835587 16430 ts_tablet_manager.cc:1428] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:01:42.835677 16486 tablet_replica.cc:330] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f: stopping tablet replica
I20260808 07:01:42.835774 16486 raft_consensus.cc:2229] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:01:42.835857 16429 tablet_bootstrap.cc:492] T 8774bb94f82e4f148ea99a2c3b353514 P 88e89354ac114e9b878e1be757114823: No bootstrap required, opened a new log
I20260808 07:01:42.835881 16486 raft_consensus.cc:2258] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:01:42.835906 16429 ts_tablet_manager.cc:1397] T 8774bb94f82e4f148ea99a2c3b353514 P 88e89354ac114e9b878e1be757114823: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:01:42.836056 16429 raft_consensus.cc:348] T 8774bb94f82e4f148ea99a2c3b353514 P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.836122 16429 raft_consensus.cc:374] T 8774bb94f82e4f148ea99a2c3b353514 P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:42.836145 16429 raft_consensus.cc:729] T 8774bb94f82e4f148ea99a2c3b353514 P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88e89354ac114e9b878e1be757114823, State: Initialized, Role: FOLLOWER
I20260808 07:01:42.836208 16429 consensus_queue.cc:260] T 8774bb94f82e4f148ea99a2c3b353514 P 88e89354ac114e9b878e1be757114823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:42.836305 16429 ts_tablet_manager.cc:1428] T 8774bb94f82e4f148ea99a2c3b353514 P 88e89354ac114e9b878e1be757114823: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:01:42.836391 16486 ts_tablet_manager.cc:1905] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:01:42.838747 16483 ts_tablet_manager.cc:1905] T aef53b649ddf4b7183f0c5d19efc958e P 88e89354ac114e9b878e1be757114823: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:01:42.838768 16486 ts_tablet_manager.cc:1918] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:01:42.838991 16486 log.cc:1196] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-0/wal/wals/aef53b649ddf4b7183f0c5d19efc958e
I20260808 07:01:42.839798 16486 ts_tablet_manager.cc:1939] T aef53b649ddf4b7183f0c5d19efc958e P 5568508efd67416d8c5a791cef3fd44f: Deleting consensus metadata
I20260808 07:01:42.840389 15899 catalog_manager.cc:4841] TS 5568508efd67416d8c5a791cef3fd44f (127.13.207.1:36077): tablet aef53b649ddf4b7183f0c5d19efc958e (table test-table [id=ed98970f1c474f8d858d2cecf076b633]) successfully deleted
I20260808 07:01:42.840839 16483 ts_tablet_manager.cc:1918] T aef53b649ddf4b7183f0c5d19efc958e P 88e89354ac114e9b878e1be757114823: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:01:42.840927 16483 log.cc:1196] T aef53b649ddf4b7183f0c5d19efc958e P 88e89354ac114e9b878e1be757114823: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/aef53b649ddf4b7183f0c5d19efc958e
I20260808 07:01:42.841198 16483 ts_tablet_manager.cc:1939] T aef53b649ddf4b7183f0c5d19efc958e P 88e89354ac114e9b878e1be757114823: Deleting consensus metadata
I20260808 07:01:42.841663 15897 catalog_manager.cc:4841] TS 88e89354ac114e9b878e1be757114823 (127.13.207.3:38209): tablet aef53b649ddf4b7183f0c5d19efc958e (table test-table [id=ed98970f1c474f8d858d2cecf076b633]) successfully deleted
I20260808 07:01:43.131742 16480 raft_consensus.cc:482] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:01:43.131875 16480 raft_consensus.cc:504] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:43.132285 16480 leader_election.cc:288] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd53f8d8a09a47d5b2ddcc7e05f098be (127.13.207.2:34543), 88e89354ac114e9b878e1be757114823 (127.13.207.3:38209)
I20260808 07:01:43.132570 16215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8774bb94f82e4f148ea99a2c3b353514" candidate_uuid: "5568508efd67416d8c5a791cef3fd44f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" is_pre_election: true
I20260808 07:01:43.132732 16356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8774bb94f82e4f148ea99a2c3b353514" candidate_uuid: "5568508efd67416d8c5a791cef3fd44f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88e89354ac114e9b878e1be757114823" is_pre_election: true
I20260808 07:01:43.132965 16356 raft_consensus.cc:2454] T 8774bb94f82e4f148ea99a2c3b353514 P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5568508efd67416d8c5a791cef3fd44f in term 0.
I20260808 07:01:43.132849 16215 raft_consensus.cc:2454] T 8774bb94f82e4f148ea99a2c3b353514 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5568508efd67416d8c5a791cef3fd44f in term 0.
I20260808 07:01:43.133222 16001 leader_election.cc:302] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5568508efd67416d8c5a791cef3fd44f, 88e89354ac114e9b878e1be757114823, dd53f8d8a09a47d5b2ddcc7e05f098be; no voters:
I20260808 07:01:43.133426 16480 raft_consensus.cc:2781] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:01:43.133504 16480 raft_consensus.cc:482] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:01:43.133530 16480 raft_consensus.cc:3037] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:43.134908 16480 raft_consensus.cc:504] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:43.135104 16480 leader_election.cc:288] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f [CANDIDATE]: Term 1 election: Requested vote from peers dd53f8d8a09a47d5b2ddcc7e05f098be (127.13.207.2:34543), 88e89354ac114e9b878e1be757114823 (127.13.207.3:38209)
I20260808 07:01:43.135286 16215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8774bb94f82e4f148ea99a2c3b353514" candidate_uuid: "5568508efd67416d8c5a791cef3fd44f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be"
I20260808 07:01:43.135311 16356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8774bb94f82e4f148ea99a2c3b353514" candidate_uuid: "5568508efd67416d8c5a791cef3fd44f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88e89354ac114e9b878e1be757114823"
I20260808 07:01:43.135398 16215 raft_consensus.cc:3037] T 8774bb94f82e4f148ea99a2c3b353514 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:43.135421 16356 raft_consensus.cc:3037] T 8774bb94f82e4f148ea99a2c3b353514 P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:43.136154 16356 raft_consensus.cc:2454] T 8774bb94f82e4f148ea99a2c3b353514 P 88e89354ac114e9b878e1be757114823 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5568508efd67416d8c5a791cef3fd44f in term 1.
I20260808 07:01:43.136323 16215 raft_consensus.cc:2454] T 8774bb94f82e4f148ea99a2c3b353514 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5568508efd67416d8c5a791cef3fd44f in term 1.
I20260808 07:01:43.136443 16001 leader_election.cc:302] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5568508efd67416d8c5a791cef3fd44f, 88e89354ac114e9b878e1be757114823; no voters:
I20260808 07:01:43.136554 16480 raft_consensus.cc:2781] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:43.136631 16480 raft_consensus.cc:686] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f [term 1 LEADER]: Becoming Leader. State: Replica: 5568508efd67416d8c5a791cef3fd44f, State: Running, Role: LEADER
I20260808 07:01:43.136694 16480 consensus_queue.cc:237] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:43.137565 15912 catalog_manager.cc:5495] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5568508efd67416d8c5a791cef3fd44f (127.13.207.1). New cstate: current_term: 1 leader_uuid: "5568508efd67416d8c5a791cef3fd44f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } health_report { overall_health: UNKNOWN } } }
I20260808 07:01:43.200085 16215 raft_consensus.cc:1261] T 8774bb94f82e4f148ea99a2c3b353514 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 1 FOLLOWER]: Refusing update from remote peer 5568508efd67416d8c5a791cef3fd44f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:01:43.200367 16480 consensus_queue.cc:1035] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:01:43.200452 16356 raft_consensus.cc:1261] T 8774bb94f82e4f148ea99a2c3b353514 P 88e89354ac114e9b878e1be757114823 [term 1 FOLLOWER]: Refusing update from remote peer 5568508efd67416d8c5a791cef3fd44f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:01:43.200973 16480 consensus_queue.cc:1035] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f [LEADER]: Connected to new peer: Peer: permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 07:01:43.207645 16462 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 07:01:44.731415 15913 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:01:44.731526 15913 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:01:44.732539 15913 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830: Sending DeleteTablet for 3 replicas of tablet 8774bb94f82e4f148ea99a2c3b353514
I20260808 07:01:44.732836 16195 tablet_service.cc:1514] Processing DeleteTablet for tablet 8774bb94f82e4f148ea99a2c3b353514 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:44 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:41366
I20260808 07:01:44.732872 16336 tablet_service.cc:1514] Processing DeleteTablet for tablet 8774bb94f82e4f148ea99a2c3b353514 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:44 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:43620
I20260808 07:01:44.732971 16050 tablet_service.cc:1514] Processing DeleteTablet for tablet 8774bb94f82e4f148ea99a2c3b353514 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:44 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:38364
I20260808 07:01:44.733184 16517 tablet_replica.cc:330] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f: stopping tablet replica
I20260808 07:01:44.733244 16515 tablet_replica.cc:330] T 8774bb94f82e4f148ea99a2c3b353514 P dd53f8d8a09a47d5b2ddcc7e05f098be: stopping tablet replica
I20260808 07:01:44.733280 16517 raft_consensus.cc:2229] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:01:44.733351 16517 raft_consensus.cc:2258] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:01:44.733654 16517 ts_tablet_manager.cc:1905] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:01:44.733963 16516 tablet_replica.cc:330] T 8774bb94f82e4f148ea99a2c3b353514 P 88e89354ac114e9b878e1be757114823: stopping tablet replica
I20260808 07:01:44.734090 16515 raft_consensus.cc:2229] T 8774bb94f82e4f148ea99a2c3b353514 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:01:44.734207 16516 raft_consensus.cc:2229] T 8774bb94f82e4f148ea99a2c3b353514 P 88e89354ac114e9b878e1be757114823 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:01:44.734302 16515 raft_consensus.cc:2258] T 8774bb94f82e4f148ea99a2c3b353514 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:01:44.734395 16516 raft_consensus.cc:2258] T 8774bb94f82e4f148ea99a2c3b353514 P 88e89354ac114e9b878e1be757114823 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:01:44.735062 16516 ts_tablet_manager.cc:1905] T 8774bb94f82e4f148ea99a2c3b353514 P 88e89354ac114e9b878e1be757114823: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:01:44.735850 16517 ts_tablet_manager.cc:1918] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 07:01:44.736003 16517 log.cc:1196] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-0/wal/wals/8774bb94f82e4f148ea99a2c3b353514
I20260808 07:01:44.736313 16517 ts_tablet_manager.cc:1939] T 8774bb94f82e4f148ea99a2c3b353514 P 5568508efd67416d8c5a791cef3fd44f: Deleting consensus metadata
I20260808 07:01:44.736691 15899 catalog_manager.cc:4841] TS 5568508efd67416d8c5a791cef3fd44f (127.13.207.1:36077): tablet 8774bb94f82e4f148ea99a2c3b353514 (table test-table [id=34ec8f48ea6544eeaa71e31868523d91]) successfully deleted
I20260808 07:01:44.736845 16515 ts_tablet_manager.cc:1905] T 8774bb94f82e4f148ea99a2c3b353514 P dd53f8d8a09a47d5b2ddcc7e05f098be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:01:44.738777 16515 ts_tablet_manager.cc:1918] T 8774bb94f82e4f148ea99a2c3b353514 P dd53f8d8a09a47d5b2ddcc7e05f098be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 07:01:44.738850 16515 log.cc:1196] T 8774bb94f82e4f148ea99a2c3b353514 P dd53f8d8a09a47d5b2ddcc7e05f098be: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-1/wal/wals/8774bb94f82e4f148ea99a2c3b353514
I20260808 07:01:44.739087 16515 ts_tablet_manager.cc:1939] T 8774bb94f82e4f148ea99a2c3b353514 P dd53f8d8a09a47d5b2ddcc7e05f098be: Deleting consensus metadata
I20260808 07:01:44.739771 16516 ts_tablet_manager.cc:1918] T 8774bb94f82e4f148ea99a2c3b353514 P 88e89354ac114e9b878e1be757114823: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 07:01:44.739851 16516 log.cc:1196] T 8774bb94f82e4f148ea99a2c3b353514 P 88e89354ac114e9b878e1be757114823: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/8774bb94f82e4f148ea99a2c3b353514
I20260808 07:01:44.740103 16516 ts_tablet_manager.cc:1939] T 8774bb94f82e4f148ea99a2c3b353514 P 88e89354ac114e9b878e1be757114823: Deleting consensus metadata
I20260808 07:01:44.740525 15898 catalog_manager.cc:4841] TS dd53f8d8a09a47d5b2ddcc7e05f098be (127.13.207.2:34543): tablet 8774bb94f82e4f148ea99a2c3b353514 (table test-table [id=34ec8f48ea6544eeaa71e31868523d91]) successfully deleted
I20260808 07:01:44.741206 15897 catalog_manager.cc:4841] TS 88e89354ac114e9b878e1be757114823 (127.13.207.3:38209): tablet 8774bb94f82e4f148ea99a2c3b353514 (table test-table [id=34ec8f48ea6544eeaa71e31868523d91]) successfully deleted
Aug 08 07:01:44 dist-test-slave-8tdl krb5kdc[15866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172504, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:01:44 dist-test-slave-8tdl krb5kdc[15866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172504, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:44.768404 16525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:44.754046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35740 (local address 127.13.207.62:38167)
0808 07:01:44.754243 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:44.754245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:44.754322 (+ 77us) server_negotiation.cc:407] Connection header received
0808 07:01:44.754447 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:44.754449 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:44.754469 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:44.754562 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:44.755307 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:44.755906 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:44.756747 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:44.756982 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:44.757063 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:44.764835 (+ 7772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:44.764837 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:44.764838 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:44.764847 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:44.767069 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:44.767550 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:44.767551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:44.767551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:44.767616 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:44.767802 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:44.767802 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:44.767802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:44.767973 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:44.768058 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:44.768249 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 07:01:44.768322 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":98,"threads_started":1}
I20260808 07:01:44.770761 15913 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35740:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:01:44.770974 15913 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:01:44.772686 16195 tablet_service.cc:1467] Processing CreateTablet for tablet b78289d64f344fe19025abe48f319185 (DEFAULT_TABLE table=test-table [id=48db45aad58d472c990d74b19d568772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:44.772723 16050 tablet_service.cc:1467] Processing CreateTablet for tablet b78289d64f344fe19025abe48f319185 (DEFAULT_TABLE table=test-table [id=48db45aad58d472c990d74b19d568772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:44.772742 16336 tablet_service.cc:1467] Processing CreateTablet for tablet b78289d64f344fe19025abe48f319185 (DEFAULT_TABLE table=test-table [id=48db45aad58d472c990d74b19d568772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:44.772858 16050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b78289d64f344fe19025abe48f319185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:44.772859 16195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b78289d64f344fe19025abe48f319185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:44.772967 16336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b78289d64f344fe19025abe48f319185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:44.774463 16527 tablet_bootstrap.cc:492] T b78289d64f344fe19025abe48f319185 P dd53f8d8a09a47d5b2ddcc7e05f098be: Bootstrap starting.
I20260808 07:01:44.774507 16528 tablet_bootstrap.cc:492] T b78289d64f344fe19025abe48f319185 P 5568508efd67416d8c5a791cef3fd44f: Bootstrap starting.
I20260808 07:01:44.774782 16526 tablet_bootstrap.cc:492] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823: Bootstrap starting.
I20260808 07:01:44.774986 16527 tablet_bootstrap.cc:654] T b78289d64f344fe19025abe48f319185 P dd53f8d8a09a47d5b2ddcc7e05f098be: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:44.775087 16528 tablet_bootstrap.cc:654] T b78289d64f344fe19025abe48f319185 P 5568508efd67416d8c5a791cef3fd44f: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:44.775250 16526 tablet_bootstrap.cc:654] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:44.775837 16528 tablet_bootstrap.cc:492] T b78289d64f344fe19025abe48f319185 P 5568508efd67416d8c5a791cef3fd44f: No bootstrap required, opened a new log
I20260808 07:01:44.775830 16527 tablet_bootstrap.cc:492] T b78289d64f344fe19025abe48f319185 P dd53f8d8a09a47d5b2ddcc7e05f098be: No bootstrap required, opened a new log
I20260808 07:01:44.775898 16527 ts_tablet_manager.cc:1397] T b78289d64f344fe19025abe48f319185 P dd53f8d8a09a47d5b2ddcc7e05f098be: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:44.775939 16528 ts_tablet_manager.cc:1397] T b78289d64f344fe19025abe48f319185 P 5568508efd67416d8c5a791cef3fd44f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:44.776052 16528 raft_consensus.cc:348] T b78289d64f344fe19025abe48f319185 P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:44.776038 16527 raft_consensus.cc:348] T b78289d64f344fe19025abe48f319185 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:44.776117 16528 raft_consensus.cc:374] T b78289d64f344fe19025abe48f319185 P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:44.776117 16527 raft_consensus.cc:374] T b78289d64f344fe19025abe48f319185 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:44.776142 16527 raft_consensus.cc:729] T b78289d64f344fe19025abe48f319185 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd53f8d8a09a47d5b2ddcc7e05f098be, State: Initialized, Role: FOLLOWER
I20260808 07:01:44.776142 16528 raft_consensus.cc:729] T b78289d64f344fe19025abe48f319185 P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5568508efd67416d8c5a791cef3fd44f, State: Initialized, Role: FOLLOWER
I20260808 07:01:44.776197 16528 consensus_queue.cc:260] T b78289d64f344fe19025abe48f319185 P 5568508efd67416d8c5a791cef3fd44f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:44.776197 16527 consensus_queue.cc:260] T b78289d64f344fe19025abe48f319185 P dd53f8d8a09a47d5b2ddcc7e05f098be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:44.776300 16527 ts_tablet_manager.cc:1428] T b78289d64f344fe19025abe48f319185 P dd53f8d8a09a47d5b2ddcc7e05f098be: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:44.776301 16528 ts_tablet_manager.cc:1428] T b78289d64f344fe19025abe48f319185 P 5568508efd67416d8c5a791cef3fd44f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:44.776901 16526 tablet_bootstrap.cc:492] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823: No bootstrap required, opened a new log
I20260808 07:01:44.776963 16526 ts_tablet_manager.cc:1397] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:01:44.777127 16526 raft_consensus.cc:348] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:44.777207 16526 raft_consensus.cc:374] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:44.777233 16526 raft_consensus.cc:729] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88e89354ac114e9b878e1be757114823, State: Initialized, Role: FOLLOWER
I20260808 07:01:44.777287 16526 consensus_queue.cc:260] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:44.777576 16526 ts_tablet_manager.cc:1428] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:01:44.810803 16533 raft_consensus.cc:482] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:01:44.810950 16533 raft_consensus.cc:504] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:44.811404 16533 leader_election.cc:288] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5568508efd67416d8c5a791cef3fd44f (127.13.207.1:36077), dd53f8d8a09a47d5b2ddcc7e05f098be (127.13.207.2:34543)
I20260808 07:01:44.815563 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:44.811794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:49689 (local address 127.13.207.2:34543)
0808 07:01:44.812054 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:44.812057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:44.812067 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:01:44.812094 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:44.812095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:44.812104 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:44.812232 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:44.812355 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:44.813072 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:44.814382 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:44.815020 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:44.815119 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:44.815132 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:44.815437 (+ 305us) server_negotiation.cc:299] Negotiation successful
0808 07:01:44.815491 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":194,"threads_started":1}
I20260808 07:01:44.815547 16534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:44.811544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:36077 (local address 127.13.207.3:50123)
0808 07:01:44.811761 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:44.811776 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:44.811836 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:44.812134 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:44.812135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:44.812138 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:44.812196 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:44.812197 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:44.812900 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:44.812901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:44.814196 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:44.814199 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:44.815185 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:44.815186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:44.815282 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:44.815286 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:01:44.815339 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 07:01:44.815390 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":130,"threads_started":1}
I20260808 07:01:44.815560 16535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:44.811584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:34543 (local address 127.13.207.3:49689)
0808 07:01:44.811950 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:44.811962 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:44.812011 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:44.812231 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:44.812232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:44.812234 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:44.812278 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:44.812279 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:44.813213 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:44.813214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:44.814297 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:44.814299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:44.815208 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:44.815209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:44.815265 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:44.815272 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 07:01:44.815339 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 07:01:44.815391 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":2,"thread_start_us":97,"threads_started":1}
I20260808 07:01:44.815835 16536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:44.811695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:50123 (local address 127.13.207.1:36077)
0808 07:01:44.811950 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:44.811952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:44.811961 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:01:44.811991 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:44.811992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:44.812004 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:44.812128 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:44.812280 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:44.812805 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:44.814363 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:44.815068 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:44.815142 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:44.815151 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:44.815698 (+ 547us) server_negotiation.cc:299] Negotiation successful
0808 07:01:44.815762 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":98,"threads_started":1}
I20260808 07:01:44.816012 16215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b78289d64f344fe19025abe48f319185" candidate_uuid: "88e89354ac114e9b878e1be757114823" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" is_pre_election: true
I20260808 07:01:44.816130 16215 raft_consensus.cc:2454] T b78289d64f344fe19025abe48f319185 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88e89354ac114e9b878e1be757114823 in term 0.
I20260808 07:01:44.816102 16073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b78289d64f344fe19025abe48f319185" candidate_uuid: "88e89354ac114e9b878e1be757114823" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5568508efd67416d8c5a791cef3fd44f" is_pre_election: true
I20260808 07:01:44.816191 16073 raft_consensus.cc:2454] T b78289d64f344fe19025abe48f319185 P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88e89354ac114e9b878e1be757114823 in term 0.
I20260808 07:01:44.816336 16288 leader_election.cc:302] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88e89354ac114e9b878e1be757114823, dd53f8d8a09a47d5b2ddcc7e05f098be; no voters:
I20260808 07:01:44.816490 16533 raft_consensus.cc:2781] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:01:44.816531 16533 raft_consensus.cc:482] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:01:44.816563 16533 raft_consensus.cc:3037] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:44.817306 16533 raft_consensus.cc:504] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:44.817484 16533 leader_election.cc:288] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823 [CANDIDATE]: Term 1 election: Requested vote from peers 5568508efd67416d8c5a791cef3fd44f (127.13.207.1:36077), dd53f8d8a09a47d5b2ddcc7e05f098be (127.13.207.2:34543)
I20260808 07:01:44.817653 16073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b78289d64f344fe19025abe48f319185" candidate_uuid: "88e89354ac114e9b878e1be757114823" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5568508efd67416d8c5a791cef3fd44f"
I20260808 07:01:44.817744 16073 raft_consensus.cc:3037] T b78289d64f344fe19025abe48f319185 P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:44.817726 16215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b78289d64f344fe19025abe48f319185" candidate_uuid: "88e89354ac114e9b878e1be757114823" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be"
I20260808 07:01:44.817796 16215 raft_consensus.cc:3037] T b78289d64f344fe19025abe48f319185 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:44.818584 16073 raft_consensus.cc:2454] T b78289d64f344fe19025abe48f319185 P 5568508efd67416d8c5a791cef3fd44f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88e89354ac114e9b878e1be757114823 in term 1.
I20260808 07:01:44.818584 16215 raft_consensus.cc:2454] T b78289d64f344fe19025abe48f319185 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88e89354ac114e9b878e1be757114823 in term 1.
I20260808 07:01:44.818826 16288 leader_election.cc:302] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88e89354ac114e9b878e1be757114823, dd53f8d8a09a47d5b2ddcc7e05f098be; no voters:
I20260808 07:01:44.818935 16533 raft_consensus.cc:2781] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:44.819180 16533 raft_consensus.cc:686] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823 [term 1 LEADER]: Becoming Leader. State: Replica: 88e89354ac114e9b878e1be757114823, State: Running, Role: LEADER
I20260808 07:01:44.819317 16533 consensus_queue.cc:237] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:44.820464 15912 catalog_manager.cc:5495] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88e89354ac114e9b878e1be757114823 (127.13.207.3). New cstate: current_term: 1 leader_uuid: "88e89354ac114e9b878e1be757114823" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } health_report { overall_health: HEALTHY } } }
I20260808 07:01:44.841578 16541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:44.838686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42716 (local address 127.13.207.3:38209)
0808 07:01:44.838973 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:44.838975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:44.839022 (+ 47us) server_negotiation.cc:407] Connection header received
0808 07:01:44.839062 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:44.839063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:44.839084 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:44.839169 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:01:44.839335 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:44.839840 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:44.840643 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:44.840846 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:44.840903 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:44.841135 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:01:44.841249 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:01:44.841321 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:44.841456 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 07:01:44.841477 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":212,"threads_started":1}
I20260808 07:01:44.847641 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:44.845056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:38307 (local address 127.13.207.2:34543)
0808 07:01:44.845112 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:44.845114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:44.845169 (+ 55us) server_negotiation.cc:407] Connection header received
0808 07:01:44.845233 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:44.845234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:44.845244 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:44.845300 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:01:44.845559 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:44.846088 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:44.847004 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:44.847218 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:44.847285 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:44.847434 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:01:44.847463 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:01:44.847504 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:44.847555 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 07:01:44.847575 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:01:44.847676 16544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:44.844874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:34543 (local address 127.13.207.3:38307)
0808 07:01:44.845113 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:44.845124 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:44.845181 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:44.845391 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:44.845393 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:44.845395 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:01:44.845479 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:44.845480 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:44.846195 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:44.846195 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:44.846890 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:44.846893 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:44.847302 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:44.847303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:44.847356 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:44.847370 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:01:44.847516 (+ 146us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:01:44.847518 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:01:44.847551 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 07:01:44.847592 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":161,"threads_started":1}
I20260808 07:01:44.850476 16215 raft_consensus.cc:1261] T b78289d64f344fe19025abe48f319185 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 1 FOLLOWER]: Refusing update from remote peer 88e89354ac114e9b878e1be757114823: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:01:44.850455 16073 raft_consensus.cc:1261] T b78289d64f344fe19025abe48f319185 P 5568508efd67416d8c5a791cef3fd44f [term 1 FOLLOWER]: Refusing update from remote peer 88e89354ac114e9b878e1be757114823: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:01:44.850826 16538 consensus_queue.cc:1035] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:01:44.850978 16533 consensus_queue.cc:1035] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:01:44.859148 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:44.856435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:38209 (local address 127.13.207.2:60387)
0808 07:01:44.856636 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:44.856649 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:44.856715 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:44.856837 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:44.856838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:44.856839 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:01:44.856891 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:44.856893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:44.857548 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:44.857549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:44.858239 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:44.858242 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:44.858730 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:44.858731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:44.858790 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:44.858796 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:01:44.858966 (+ 170us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:01:44.858968 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:01:44.859020 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:01:44.859069 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":128,"threads_started":1}
I20260808 07:01:44.859140 16541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:44.856622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:60387 (local address 127.13.207.3:38209)
0808 07:01:44.856670 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:44.856672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:44.856697 (+ 25us) server_negotiation.cc:407] Connection header received
0808 07:01:44.856770 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:44.856771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:44.856782 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:44.856826 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:01:44.856957 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:44.857448 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:44.858313 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:44.858601 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:44.858696 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:44.858867 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:01:44.858904 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:01:44.858956 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:44.859029 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 07:01:44.859051 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:01:44.869445 15913 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:01:44.869570 15913 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:01:44.870262 15913 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830: Sending DeleteTablet for 3 replicas of tablet b78289d64f344fe19025abe48f319185
I20260808 07:01:44.870574 16050 tablet_service.cc:1514] Processing DeleteTablet for tablet b78289d64f344fe19025abe48f319185 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:44 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:38364
I20260808 07:01:44.870740 16517 tablet_replica.cc:330] T b78289d64f344fe19025abe48f319185 P 5568508efd67416d8c5a791cef3fd44f: stopping tablet replica
I20260808 07:01:44.870803 16517 raft_consensus.cc:2229] T b78289d64f344fe19025abe48f319185 P 5568508efd67416d8c5a791cef3fd44f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:01:44.870855 16517 raft_consensus.cc:2258] T b78289d64f344fe19025abe48f319185 P 5568508efd67416d8c5a791cef3fd44f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:01:44.871044 16336 tablet_service.cc:1514] Processing DeleteTablet for tablet b78289d64f344fe19025abe48f319185 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:44 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:43620
I20260808 07:01:44.871158 16516 tablet_replica.cc:330] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823: stopping tablet replica
I20260808 07:01:44.871219 16516 raft_consensus.cc:2229] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:01:44.871313 16516 raft_consensus.cc:2258] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:01:44.871619 16195 tablet_service.cc:1514] Processing DeleteTablet for tablet b78289d64f344fe19025abe48f319185 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:44 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:41366
I20260808 07:01:44.871752 16515 tablet_replica.cc:330] T b78289d64f344fe19025abe48f319185 P dd53f8d8a09a47d5b2ddcc7e05f098be: stopping tablet replica
I20260808 07:01:44.871824 16516 ts_tablet_manager.cc:1905] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:01:44.871847 16515 raft_consensus.cc:2229] T b78289d64f344fe19025abe48f319185 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:01:44.871901 16515 raft_consensus.cc:2258] T b78289d64f344fe19025abe48f319185 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:01:44.871860 15913 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35740:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:01:44.872047 15913 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:01:44.872290 16515 ts_tablet_manager.cc:1905] T b78289d64f344fe19025abe48f319185 P dd53f8d8a09a47d5b2ddcc7e05f098be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:01:44.873546 16336 tablet_service.cc:1467] Processing CreateTablet for tablet 331f51835d0842d2bec9133ce8fa9a96 (DEFAULT_TABLE table=test-table [id=5a79cb50c80644d89f89b97cb1059802]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:44.873574 16050 tablet_service.cc:1467] Processing CreateTablet for tablet 331f51835d0842d2bec9133ce8fa9a96 (DEFAULT_TABLE table=test-table [id=5a79cb50c80644d89f89b97cb1059802]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:44.873685 16050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 331f51835d0842d2bec9133ce8fa9a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:44.873687 16336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 331f51835d0842d2bec9133ce8fa9a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:44.874154 16517 ts_tablet_manager.cc:1905] T b78289d64f344fe19025abe48f319185 P 5568508efd67416d8c5a791cef3fd44f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:01:44.874414 16516 ts_tablet_manager.cc:1918] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:01:44.874420 16515 ts_tablet_manager.cc:1918] T b78289d64f344fe19025abe48f319185 P dd53f8d8a09a47d5b2ddcc7e05f098be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:01:44.874469 16516 log.cc:1196] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/b78289d64f344fe19025abe48f319185
I20260808 07:01:44.874476 16515 log.cc:1196] T b78289d64f344fe19025abe48f319185 P dd53f8d8a09a47d5b2ddcc7e05f098be: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-1/wal/wals/b78289d64f344fe19025abe48f319185
I20260808 07:01:44.874739 16515 ts_tablet_manager.cc:1939] T b78289d64f344fe19025abe48f319185 P dd53f8d8a09a47d5b2ddcc7e05f098be: Deleting consensus metadata
I20260808 07:01:44.875206 16528 tablet_bootstrap.cc:492] T 331f51835d0842d2bec9133ce8fa9a96 P 5568508efd67416d8c5a791cef3fd44f: Bootstrap starting.
I20260808 07:01:44.875540 16516 ts_tablet_manager.cc:1939] T b78289d64f344fe19025abe48f319185 P 88e89354ac114e9b878e1be757114823: Deleting consensus metadata
I20260808 07:01:44.875586 16528 tablet_bootstrap.cc:654] T 331f51835d0842d2bec9133ce8fa9a96 P 5568508efd67416d8c5a791cef3fd44f: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:44.875875 16526 tablet_bootstrap.cc:492] T 331f51835d0842d2bec9133ce8fa9a96 P 88e89354ac114e9b878e1be757114823: Bootstrap starting.
I20260808 07:01:44.876214 16517 ts_tablet_manager.cc:1918] T b78289d64f344fe19025abe48f319185 P 5568508efd67416d8c5a791cef3fd44f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:01:44.876266 16517 log.cc:1196] T b78289d64f344fe19025abe48f319185 P 5568508efd67416d8c5a791cef3fd44f: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-0/wal/wals/b78289d64f344fe19025abe48f319185
I20260808 07:01:44.876289 16526 tablet_bootstrap.cc:654] T 331f51835d0842d2bec9133ce8fa9a96 P 88e89354ac114e9b878e1be757114823: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:44.876295 16528 tablet_bootstrap.cc:492] T 331f51835d0842d2bec9133ce8fa9a96 P 5568508efd67416d8c5a791cef3fd44f: No bootstrap required, opened a new log
I20260808 07:01:44.876346 16528 ts_tablet_manager.cc:1397] T 331f51835d0842d2bec9133ce8fa9a96 P 5568508efd67416d8c5a791cef3fd44f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:44.876543 16517 ts_tablet_manager.cc:1939] T b78289d64f344fe19025abe48f319185 P 5568508efd67416d8c5a791cef3fd44f: Deleting consensus metadata
I20260808 07:01:44.876523 16528 raft_consensus.cc:348] T 331f51835d0842d2bec9133ce8fa9a96 P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:44.876605 16528 raft_consensus.cc:374] T 331f51835d0842d2bec9133ce8fa9a96 P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:44.876628 16528 raft_consensus.cc:729] T 331f51835d0842d2bec9133ce8fa9a96 P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5568508efd67416d8c5a791cef3fd44f, State: Initialized, Role: FOLLOWER
I20260808 07:01:44.876662 16528 consensus_queue.cc:260] T 331f51835d0842d2bec9133ce8fa9a96 P 5568508efd67416d8c5a791cef3fd44f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:44.876765 16528 ts_tablet_manager.cc:1428] T 331f51835d0842d2bec9133ce8fa9a96 P 5568508efd67416d8c5a791cef3fd44f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:01:44.876883 15897 catalog_manager.cc:4841] TS 88e89354ac114e9b878e1be757114823 (127.13.207.3:38209): tablet b78289d64f344fe19025abe48f319185 (table test-table [id=48db45aad58d472c990d74b19d568772]) successfully deleted
I20260808 07:01:44.877341 16526 tablet_bootstrap.cc:492] T 331f51835d0842d2bec9133ce8fa9a96 P 88e89354ac114e9b878e1be757114823: No bootstrap required, opened a new log
I20260808 07:01:44.877419 16526 ts_tablet_manager.cc:1397] T 331f51835d0842d2bec9133ce8fa9a96 P 88e89354ac114e9b878e1be757114823: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:01:44.877589 16526 raft_consensus.cc:348] T 331f51835d0842d2bec9133ce8fa9a96 P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:44.877667 16526 raft_consensus.cc:374] T 331f51835d0842d2bec9133ce8fa9a96 P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:44.877699 16526 raft_consensus.cc:729] T 331f51835d0842d2bec9133ce8fa9a96 P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88e89354ac114e9b878e1be757114823, State: Initialized, Role: FOLLOWER
I20260808 07:01:44.877750 16526 consensus_queue.cc:260] T 331f51835d0842d2bec9133ce8fa9a96 P 88e89354ac114e9b878e1be757114823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:44.877867 16526 ts_tablet_manager.cc:1428] T 331f51835d0842d2bec9133ce8fa9a96 P 88e89354ac114e9b878e1be757114823: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:01:44.878773 15898 catalog_manager.cc:4841] TS dd53f8d8a09a47d5b2ddcc7e05f098be (127.13.207.2:34543): tablet b78289d64f344fe19025abe48f319185 (table test-table [id=48db45aad58d472c990d74b19d568772]) successfully deleted
I20260808 07:01:44.879048 16195 tablet_service.cc:1467] Processing CreateTablet for tablet 331f51835d0842d2bec9133ce8fa9a96 (DEFAULT_TABLE table=test-table [id=5a79cb50c80644d89f89b97cb1059802]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:44.879195 16195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 331f51835d0842d2bec9133ce8fa9a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:44.879421 15899 catalog_manager.cc:4841] TS 5568508efd67416d8c5a791cef3fd44f (127.13.207.1:36077): tablet b78289d64f344fe19025abe48f319185 (table test-table [id=48db45aad58d472c990d74b19d568772]) successfully deleted
I20260808 07:01:44.880313 16527 tablet_bootstrap.cc:492] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be: Bootstrap starting.
I20260808 07:01:44.880697 16527 tablet_bootstrap.cc:654] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:44.881381 16527 tablet_bootstrap.cc:492] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be: No bootstrap required, opened a new log
I20260808 07:01:44.881446 16527 ts_tablet_manager.cc:1397] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:44.881589 16527 raft_consensus.cc:348] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:44.881683 16527 raft_consensus.cc:374] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:44.881706 16527 raft_consensus.cc:729] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd53f8d8a09a47d5b2ddcc7e05f098be, State: Initialized, Role: FOLLOWER
I20260808 07:01:44.881795 16527 consensus_queue.cc:260] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:44.881901 16527 ts_tablet_manager.cc:1428] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:01:45.038650 16477 raft_consensus.cc:482] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:01:45.038797 16477 raft_consensus.cc:504] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:45.038992 16477 leader_election.cc:288] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5568508efd67416d8c5a791cef3fd44f (127.13.207.1:36077), 88e89354ac114e9b878e1be757114823 (127.13.207.3:38209)
I20260808 07:01:45.039239 16073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331f51835d0842d2bec9133ce8fa9a96" candidate_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5568508efd67416d8c5a791cef3fd44f" is_pre_election: true
I20260808 07:01:45.039352 16073 raft_consensus.cc:2454] T 331f51835d0842d2bec9133ce8fa9a96 P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd53f8d8a09a47d5b2ddcc7e05f098be in term 0.
I20260808 07:01:45.039309 16356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331f51835d0842d2bec9133ce8fa9a96" candidate_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88e89354ac114e9b878e1be757114823" is_pre_election: true
I20260808 07:01:45.039444 16356 raft_consensus.cc:2454] T 331f51835d0842d2bec9133ce8fa9a96 P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd53f8d8a09a47d5b2ddcc7e05f098be in term 0.
I20260808 07:01:45.039572 16146 leader_election.cc:302] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5568508efd67416d8c5a791cef3fd44f, dd53f8d8a09a47d5b2ddcc7e05f098be; no voters:
I20260808 07:01:45.039717 16477 raft_consensus.cc:2781] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:01:45.039790 16477 raft_consensus.cc:482] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:01:45.039826 16477 raft_consensus.cc:3037] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:45.040671 16477 raft_consensus.cc:504] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:45.040936 16477 leader_election.cc:288] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be [CANDIDATE]: Term 1 election: Requested vote from peers 5568508efd67416d8c5a791cef3fd44f (127.13.207.1:36077), 88e89354ac114e9b878e1be757114823 (127.13.207.3:38209)
I20260808 07:01:45.041225 16073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331f51835d0842d2bec9133ce8fa9a96" candidate_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5568508efd67416d8c5a791cef3fd44f"
I20260808 07:01:45.041225 16356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331f51835d0842d2bec9133ce8fa9a96" candidate_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88e89354ac114e9b878e1be757114823"
I20260808 07:01:45.041334 16073 raft_consensus.cc:3037] T 331f51835d0842d2bec9133ce8fa9a96 P 5568508efd67416d8c5a791cef3fd44f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:45.041334 16356 raft_consensus.cc:3037] T 331f51835d0842d2bec9133ce8fa9a96 P 88e89354ac114e9b878e1be757114823 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:45.042176 16073 raft_consensus.cc:2454] T 331f51835d0842d2bec9133ce8fa9a96 P 5568508efd67416d8c5a791cef3fd44f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd53f8d8a09a47d5b2ddcc7e05f098be in term 1.
I20260808 07:01:45.042187 16356 raft_consensus.cc:2454] T 331f51835d0842d2bec9133ce8fa9a96 P 88e89354ac114e9b878e1be757114823 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd53f8d8a09a47d5b2ddcc7e05f098be in term 1.
I20260808 07:01:45.042615 16144 leader_election.cc:302] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88e89354ac114e9b878e1be757114823, dd53f8d8a09a47d5b2ddcc7e05f098be; no voters:
I20260808 07:01:45.042775 16477 raft_consensus.cc:2781] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:45.042876 16477 raft_consensus.cc:686] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 1 LEADER]: Becoming Leader. State: Replica: dd53f8d8a09a47d5b2ddcc7e05f098be, State: Running, Role: LEADER
I20260808 07:01:45.042928 16477 consensus_queue.cc:237] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } }
I20260808 07:01:45.043838 15912 catalog_manager.cc:5495] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be reported cstate change: term changed from 0 to 1, leader changed from <none> to dd53f8d8a09a47d5b2ddcc7e05f098be (127.13.207.2). New cstate: current_term: 1 leader_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd53f8d8a09a47d5b2ddcc7e05f098be" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 } health_report { overall_health: UNKNOWN } } }
I20260808 07:01:45.070992 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:45.067885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51784 (local address 127.13.207.2:34543)
0808 07:01:45.068001 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:45.068003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:45.068012 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:01:45.068034 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:45.068034 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:45.068043 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:45.068094 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:01:45.068320 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:45.068843 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:45.069772 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:45.070162 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:45.070276 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:45.070567 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:01:45.070599 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:01:45.070647 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:45.070856 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 07:01:45.070881 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 07:01:45.077369 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:45.074263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:34543 (local address 127.13.207.2:44053)
0808 07:01:45.074326 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:45.074338 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:45.074379 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:45.074834 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:45.074836 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:45.074837 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:01:45.074893 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:45.074895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:45.075715 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:45.075716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:45.076403 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:45.076412 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:45.076798 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:45.076799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:45.076906 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:45.076922 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:01:45.077188 (+ 266us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:01:45.077190 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:01:45.077241 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 07:01:45.077305 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":8}
I20260808 07:01:45.077369 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:45.074460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:44053 (local address 127.13.207.2:34543)
0808 07:01:45.074629 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:45.074631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:45.074640 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:01:45.074659 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:45.074659 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:45.074669 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:45.074830 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:01:45.074978 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:45.075602 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:45.076526 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:45.076714 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:45.076799 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:45.077083 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:01:45.077112 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:01:45.077188 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:45.077242 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 07:01:45.077269 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":146}
I20260808 07:01:45.080500 16356 raft_consensus.cc:1261] T 331f51835d0842d2bec9133ce8fa9a96 P 88e89354ac114e9b878e1be757114823 [term 1 FOLLOWER]: Refusing update from remote peer dd53f8d8a09a47d5b2ddcc7e05f098be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:01:45.080492 16073 raft_consensus.cc:1261] T 331f51835d0842d2bec9133ce8fa9a96 P 5568508efd67416d8c5a791cef3fd44f [term 1 FOLLOWER]: Refusing update from remote peer dd53f8d8a09a47d5b2ddcc7e05f098be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:01:45.080765 16477 consensus_queue.cc:1035] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be [LEADER]: Connected to new peer: Peer: permanent_uuid: "5568508efd67416d8c5a791cef3fd44f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:01:45.080897 16496 consensus_queue.cc:1035] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be [LEADER]: Connected to new peer: Peer: permanent_uuid: "88e89354ac114e9b878e1be757114823" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 38209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 07:01:45.087193 16494 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 07:01:46.609691 15911 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:01:46.609804 15911 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:01:46.610910 15911 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be83e223859f4e59b15d2d5421664830: Sending DeleteTablet for 3 replicas of tablet 331f51835d0842d2bec9133ce8fa9a96
I20260808 07:01:46.611258 16050 tablet_service.cc:1514] Processing DeleteTablet for tablet 331f51835d0842d2bec9133ce8fa9a96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:46 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:38364
I20260808 07:01:46.611312 16336 tablet_service.cc:1514] Processing DeleteTablet for tablet 331f51835d0842d2bec9133ce8fa9a96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:46 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:43620
I20260808 07:01:46.611514 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 15985
I20260808 07:01:46.611555 16194 tablet_service.cc:1514] Processing DeleteTablet for tablet 331f51835d0842d2bec9133ce8fa9a96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:46 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:41366
I20260808 07:01:46.618893 16582 tablet_replica.cc:330] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be: stopping tablet replica
I20260808 07:01:46.618999 16582 raft_consensus.cc:2229] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:01:46.619143 16582 raft_consensus.cc:2258] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:01:46.619160 16581 tablet_replica.cc:330] T 331f51835d0842d2bec9133ce8fa9a96 P 88e89354ac114e9b878e1be757114823: stopping tablet replica
I20260808 07:01:46.619251 16581 raft_consensus.cc:2229] T 331f51835d0842d2bec9133ce8fa9a96 P 88e89354ac114e9b878e1be757114823 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:01:46.619359 16581 raft_consensus.cc:2258] T 331f51835d0842d2bec9133ce8fa9a96 P 88e89354ac114e9b878e1be757114823 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:01:46.619582 16582 ts_tablet_manager.cc:1905] T 331f51835d0842d2bec9133ce8fa9a96 P dd53f8d8a09a47d5b2ddcc7e05f098be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:01:46.619681 16581 ts_tablet_manager.cc:1905] T 331f51835d0842d2bec9133ce8fa9a96 P 88e89354ac114e9b878e1be757114823: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 07:01:46.620342 15899 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.207.1:36077 (error 108)
I20260808 07:01:46.620752 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 16128
I20260808 07:01:46.622346 16581 ts_tablet_manager.cc:1918] T 331f51835d0842d2bec9133ce8fa9a96 P 88e89354ac114e9b878e1be757114823: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 07:01:46.622459 16581 log.cc:1196] T 331f51835d0842d2bec9133ce8fa9a96 P 88e89354ac114e9b878e1be757114823: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/331f51835d0842d2bec9133ce8fa9a96
I20260808 07:01:46.622759 16581 ts_tablet_manager.cc:1939] T 331f51835d0842d2bec9133ce8fa9a96 P 88e89354ac114e9b878e1be757114823: Deleting consensus metadata
I20260808 07:01:46.623692 15897 catalog_manager.cc:4841] TS 88e89354ac114e9b878e1be757114823 (127.13.207.3:38209): tablet 331f51835d0842d2bec9133ce8fa9a96 (table test-table [id=5a79cb50c80644d89f89b97cb1059802]) successfully deleted
I20260808 07:01:46.623785 16584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:46.623441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:36077 (local address 127.0.0.1:47214)
0808 07:01:46.623621 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:46.623693 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.1:36077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":93,"threads_started":1}
W20260808 07:01:46.623947 15899 catalog_manager.cc:4584] TS 5568508efd67416d8c5a791cef3fd44f (127.13.207.1:36077): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 331f51835d0842d2bec9133ce8fa9a96: Network error: Client connection negotiation failed: client connection to 127.13.207.1:36077: connect: Connection refused (error 111)
I20260808 07:01:46.628969 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 16270
I20260808 07:01:46.629076 16584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:46.628838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:34543 (local address 127.0.0.1:51788)
0808 07:01:46.628912 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:46.629029 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.2:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 07:01:46.637445 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 15880
2026-08-08T07:01:46Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6228 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:01:46 dist-test-slave-8tdl krb5kdc[16587](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:01:46 dist-test-slave-8tdl krb5kdc[16587](info): set up 2 sockets
Aug 08 07:01:46 dist-test-slave-8tdl krb5kdc[16587](info): commencing operation
krb5kdc: starting...
W20260808 07:01:47.775621 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:01:47 dist-test-slave-8tdl krb5kdc[16587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:01:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:01:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:01:47.928638 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:40627
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:40627
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:48.023981 16600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:48.024298 16600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:48.024345 16600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:48.025748 16600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:01:48.025841 16600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:48.025866 16600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:48.025887 16600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:01:48.025915 16600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:01:48.028095 16600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:40627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:40627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:48.028507 16600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:48.028815 16600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:01:48.035353 16600 server_base.cc:1034] running on GCE node
W20260808 07:01:48.035341 16605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:48.035377 16608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:48.035368 16606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:48.036106 16600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:48.036465 16600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:48.037649 16600 hybrid_clock.cc:648] HybridClock initialized: now 1786172508037628 us; error 40 us; skew 500 ppm
Aug 08 07:01:48 dist-test-slave-8tdl krb5kdc[16587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172508, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:48.040899 16600 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:01:48.041462 16600 webserver.cc:460] Webserver started at http://127.13.207.62:38701/ using document root <none> and password file <none>
I20260808 07:01:48.041791 16600 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:48.041872 16600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:48.042042 16600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:48.043334 16600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "ca7547ba37e44a6795f4a76affd805c5"
format_stamp: "Formatted at 2026-08-08 07:01:48 on dist-test-slave-8tdl"
server_key: "ac36a11c2a3504163ccfc36a4762a81d"
server_key_iv: "722bac1d2f910285566e8a32c257be64"
server_key_version: "encryptionkey@0"
I20260808 07:01:48.043740 16600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "ca7547ba37e44a6795f4a76affd805c5"
format_stamp: "Formatted at 2026-08-08 07:01:48 on dist-test-slave-8tdl"
server_key: "ac36a11c2a3504163ccfc36a4762a81d"
server_key_iv: "722bac1d2f910285566e8a32c257be64"
server_key_version: "encryptionkey@0"
I20260808 07:01:48.045570 16600 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 07:01:48.046669 16615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:48.047009 16600 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:48.047144 16600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "ca7547ba37e44a6795f4a76affd805c5"
format_stamp: "Formatted at 2026-08-08 07:01:48 on dist-test-slave-8tdl"
server_key: "ac36a11c2a3504163ccfc36a4762a81d"
server_key_iv: "722bac1d2f910285566e8a32c257be64"
server_key_version: "encryptionkey@0"
I20260808 07:01:48.047243 16600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:48.070398 16600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:48.073042 16600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:48.073252 16600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:48.078404 16600 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:40627
I20260808 07:01:48.078505 16670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:40627 every 8 connection(s)
I20260808 07:01:48.078950 16600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:01:48.080041 16671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:48.082824 16671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca7547ba37e44a6795f4a76affd805c5: Bootstrap starting.
I20260808 07:01:48.083500 16671 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca7547ba37e44a6795f4a76affd805c5: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:48.083781 16671 log.cc:824] T 00000000000000000000000000000000 P ca7547ba37e44a6795f4a76affd805c5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:48.084717 16671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca7547ba37e44a6795f4a76affd805c5: No bootstrap required, opened a new log
I20260808 07:01:48.086122 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 16600
I20260808 07:01:48.086244 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:01:48.086440 14140 external_mini_cluster.cc:1428] Setting key 861c8b36001f2e3c16e5e9406d488237
I20260808 07:01:48.086740 16671 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca7547ba37e44a6795f4a76affd805c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7547ba37e44a6795f4a76affd805c5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40627 } }
I20260808 07:01:48.087016 16671 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca7547ba37e44a6795f4a76affd805c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:48.087078 16671 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca7547ba37e44a6795f4a76affd805c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca7547ba37e44a6795f4a76affd805c5, State: Initialized, Role: FOLLOWER
I20260808 07:01:48.087265 16671 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca7547ba37e44a6795f4a76affd805c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7547ba37e44a6795f4a76affd805c5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40627 } }
I20260808 07:01:48.087407 16671 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca7547ba37e44a6795f4a76affd805c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:01:48.087461 16671 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca7547ba37e44a6795f4a76affd805c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:01:48.087522 16671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca7547ba37e44a6795f4a76affd805c5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:48.088565 16671 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca7547ba37e44a6795f4a76affd805c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7547ba37e44a6795f4a76affd805c5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40627 } }
I20260808 07:01:48.088708 16671 leader_election.cc:302] T 00000000000000000000000000000000 P ca7547ba37e44a6795f4a76affd805c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca7547ba37e44a6795f4a76affd805c5; no voters:
I20260808 07:01:48.088958 16671 leader_election.cc:288] T 00000000000000000000000000000000 P ca7547ba37e44a6795f4a76affd805c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:01:48.088996 16677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca7547ba37e44a6795f4a76affd805c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:48.089152 16677 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca7547ba37e44a6795f4a76affd805c5 [term 1 LEADER]: Becoming Leader. State: Replica: ca7547ba37e44a6795f4a76affd805c5, State: Running, Role: LEADER
I20260808 07:01:48.089279 16671 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca7547ba37e44a6795f4a76affd805c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:01:48.089325 16677 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca7547ba37e44a6795f4a76affd805c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7547ba37e44a6795f4a76affd805c5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40627 } }
I20260808 07:01:48.089705 16679 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca7547ba37e44a6795f4a76affd805c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca7547ba37e44a6795f4a76affd805c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7547ba37e44a6795f4a76affd805c5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40627 } } }
I20260808 07:01:48.089686 16677 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca7547ba37e44a6795f4a76affd805c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca7547ba37e44a6795f4a76affd805c5. Latest consensus state: current_term: 1 leader_uuid: "ca7547ba37e44a6795f4a76affd805c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7547ba37e44a6795f4a76affd805c5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40627 } } }
I20260808 07:01:48.089900 16679 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca7547ba37e44a6795f4a76affd805c5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:48.089919 16677 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca7547ba37e44a6795f4a76affd805c5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:48.091421 16687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:01:48.092772 16687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:01:48.094916 16687 catalog_manager.cc:1347] Generated new cluster ID: a7c5ded8a3b5484e86b483d680b32af8
I20260808 07:01:48.095010 16687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:01:48 dist-test-slave-8tdl krb5kdc[16587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:48.100183 16675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:48.087142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40386 (local address 127.13.207.62:40627)
0808 07:01:48.087391 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:48.087403 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:48.087569 (+ 166us) server_negotiation.cc:407] Connection header received
0808 07:01:48.087714 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:48.087720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:48.087767 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:48.087908 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:48.088566 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:48.090418 (+ 1852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:48.091299 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:48.092001 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:48.092123 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:48.095801 (+ 3678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:48.095806 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:48.095806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:48.095819 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:48.098102 (+ 2283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:48.098718 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:48.098721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:48.098722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:48.098864 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:48.099115 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:48.099115 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:48.099116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:48.099445 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:48.099615 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:48.099746 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 07:01:48.099921 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":122,"threads_started":1}
I20260808 07:01:48.104908 16687 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:01:48.105798 16687 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:01:48.126427 16687 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca7547ba37e44a6795f4a76affd805c5: Generated new TSK 0
I20260808 07:01:48.126681 16687 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
I20260808 07:01:48.178007 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40627
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:48.290560 16699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:48.290908 16699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:48.290942 16699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:48.292238 16699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:48.292289 16699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:48.292320 16699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:01:48.294533 16699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:48.294898 16699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:48.295257 16699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:48.301728 16705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:48.301862 16707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:48.301831 16704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:48.301748 16699 server_base.cc:1034] running on GCE node
I20260808 07:01:48.302276 16699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:48.302562 16699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:48.303740 16699 hybrid_clock.cc:648] HybridClock initialized: now 1786172508303677 us; error 86 us; skew 500 ppm
Aug 08 07:01:48 dist-test-slave-8tdl krb5kdc[16587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172508, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:48.307013 16699 init.cc:377] Logged in from keytab as kudu/127.13.207.1@KRBTEST.COM (short username kudu)
I20260808 07:01:48.307585 16699 webserver.cc:460] Webserver started at http://127.13.207.1:34251/ using document root <none> and password file <none>
I20260808 07:01:48.307829 16699 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:48.307878 16699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:48.308029 16699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:48.309166 16699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "e2d3b0c99afd4a7db8c97ee5accfc442"
format_stamp: "Formatted at 2026-08-08 07:01:48 on dist-test-slave-8tdl"
server_key: "f3778e0bb35b56a9e0e9cee45c89020a"
server_key_iv: "c71a2ecf0838daea5d9ba84c42e43bcb"
server_key_version: "encryptionkey@0"
I20260808 07:01:48.309548 16699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "e2d3b0c99afd4a7db8c97ee5accfc442"
format_stamp: "Formatted at 2026-08-08 07:01:48 on dist-test-slave-8tdl"
server_key: "f3778e0bb35b56a9e0e9cee45c89020a"
server_key_iv: "c71a2ecf0838daea5d9ba84c42e43bcb"
server_key_version: "encryptionkey@0"
I20260808 07:01:48.311055 16699 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:01:48.312043 16714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:48.312354 16699 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:01:48.312512 16699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "e2d3b0c99afd4a7db8c97ee5accfc442"
format_stamp: "Formatted at 2026-08-08 07:01:48 on dist-test-slave-8tdl"
server_key: "f3778e0bb35b56a9e0e9cee45c89020a"
server_key_iv: "c71a2ecf0838daea5d9ba84c42e43bcb"
server_key_version: "encryptionkey@0"
I20260808 07:01:48.312606 16699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:48.327010 16699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:48.329708 16699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:48.329909 16699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:48.330204 16699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:48.330575 16699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:48.330634 16699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:48.330675 16699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:48.330741 16699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:48.337572 16699 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:33763
I20260808 07:01:48.337641 16831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:33763 every 8 connection(s)
I20260808 07:01:48.338083 16699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:01:48 dist-test-slave-8tdl krb5kdc[16587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172508, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:48.346160 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 16699
I20260808 07:01:48.346262 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:01:48.346472 14140 external_mini_cluster.cc:1428] Setting key d95da42199717c83cac3e4ce76a32820
I20260808 07:01:48.349642 16675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:48.339022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:49393 (local address 127.13.207.62:40627)
0808 07:01:48.339098 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:48.339101 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:48.339351 (+ 250us) server_negotiation.cc:407] Connection header received
0808 07:01:48.340165 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:48.340167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:48.340184 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:48.340232 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:48.341259 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:48.341788 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:48.342916 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:48.343130 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:48.343210 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:48.345884 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:48.345886 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:48.345886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:48.345895 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:48.347858 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:48.348385 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:48.348387 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:48.348387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:48.348457 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:48.348773 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:48.348774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:48.348775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:48.348908 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:48.349068 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:48.349450 (+ 382us) server_negotiation.cc:299] Negotiation successful
0808 07:01:48.349568 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 07:01:48.350112 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:48.338989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:40627 (local address 127.13.207.1:49393)
0808 07:01:48.339204 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:48.339260 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:48.340032 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:48.340316 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:48.340320 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:48.340926 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:48.341147 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:48.341153 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:48.341903 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:48.341904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:48.342813 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:48.342816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:48.343223 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:48.343224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:48.343366 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:48.343891 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:48.343900 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:48.345768 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:48.347973 (+ 2205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:48.347986 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:48.347988 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:48.348279 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:48.348553 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:48.348554 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:48.348554 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:48.348691 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:48.349055 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:48.349064 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:48.349236 (+ 172us) client_negotiation.cc:769] Sending connection context
0808 07:01:48.349389 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 07:01:48.349632 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":111,"threads_started":1}
I20260808 07:01:48.350840 16832 heartbeater.cc:347] Connected to a master server at 127.13.207.62:40627
I20260808 07:01:48.350976 16832 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:48.351212 16832 heartbeater.cc:511] Master 127.13.207.62:40627 requested a full tablet report, sending...
I20260808 07:01:48.351826 16632 ts_manager.cc:194] Registered new tserver with Master: e2d3b0c99afd4a7db8c97ee5accfc442 (127.13.207.1:33763)
I20260808 07:01:48.352564 16632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:49393
WARNING: no policy specified for kudu/127.13.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
I20260808 07:01:48.383782 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40627
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:48.472402 16837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:48.472666 16837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:48.472692 16837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:48.474026 16837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:48.474090 16837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:48.474123 16837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:01:48.475781 16837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:48.476094 16837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:48.476351 16837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:48.482077 16846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:48.482048 16843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:48.482038 16842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:48.482326 16837 server_base.cc:1034] running on GCE node
I20260808 07:01:48.482637 16837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:48.483062 16837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:48.484308 16837 hybrid_clock.cc:648] HybridClock initialized: now 1786172508484259 us; error 61 us; skew 500 ppm
Aug 08 07:01:48 dist-test-slave-8tdl krb5kdc[16587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172508, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:48.487287 16837 init.cc:377] Logged in from keytab as kudu/127.13.207.2@KRBTEST.COM (short username kudu)
I20260808 07:01:48.487983 16837 webserver.cc:460] Webserver started at http://127.13.207.2:45991/ using document root <none> and password file <none>
I20260808 07:01:48.488286 16837 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:48.488340 16837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:48.488562 16837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:48.489970 16837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "727529a1f3114a129dbbb84b17e46f9c"
format_stamp: "Formatted at 2026-08-08 07:01:48 on dist-test-slave-8tdl"
server_key: "fe0370853c50e817cc8d2bb94cabc4bf"
server_key_iv: "436ca186fb30cb6891ed0c6a4b722f06"
server_key_version: "encryptionkey@0"
I20260808 07:01:48.490551 16837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "727529a1f3114a129dbbb84b17e46f9c"
format_stamp: "Formatted at 2026-08-08 07:01:48 on dist-test-slave-8tdl"
server_key: "fe0370853c50e817cc8d2bb94cabc4bf"
server_key_iv: "436ca186fb30cb6891ed0c6a4b722f06"
server_key_version: "encryptionkey@0"
I20260808 07:01:48.492437 16837 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260808 07:01:48.493503 16853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:48.493716 16837 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:48.493815 16837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "727529a1f3114a129dbbb84b17e46f9c"
format_stamp: "Formatted at 2026-08-08 07:01:48 on dist-test-slave-8tdl"
server_key: "fe0370853c50e817cc8d2bb94cabc4bf"
server_key_iv: "436ca186fb30cb6891ed0c6a4b722f06"
server_key_version: "encryptionkey@0"
I20260808 07:01:48.493880 16837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:48.509725 16837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:48.512595 16837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:48.512739 16837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:48.513199 16837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:48.513518 16837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:48.513554 16837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:48.513579 16837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:48.513592 16837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:48.520776 16837 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:46709
I20260808 07:01:48.520893 16971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:46709 every 8 connection(s)
I20260808 07:01:48.521672 16837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:01:48 dist-test-slave-8tdl krb5kdc[16587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172508, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:48.531046 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 16837
I20260808 07:01:48.531181 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:01:48.531445 14140 external_mini_cluster.cc:1428] Setting key d4295aaf167ac23de6a701936681ee95
I20260808 07:01:48.532984 16675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:48.522744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:40607 (local address 127.13.207.62:40627)
0808 07:01:48.522864 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:48.522866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:48.523071 (+ 205us) server_negotiation.cc:407] Connection header received
0808 07:01:48.523633 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:48.523634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:48.523647 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:48.523722 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:48.524759 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:48.525289 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:48.526337 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:48.526552 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:48.526647 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:48.529418 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:48.529421 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:48.529421 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:48.529432 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:48.531103 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:48.531668 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:48.531669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:48.531670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:48.531751 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:48.532089 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:48.532090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:48.532090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:48.532267 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:48.532358 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:48.532830 (+ 472us) server_negotiation.cc:299] Negotiation successful
0808 07:01:48.532915 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 07:01:48.533263 16974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:48.522630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:40627 (local address 127.13.207.2:40607)
0808 07:01:48.522893 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:48.522949 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:48.523519 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:48.523798 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:48.523803 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:48.524429 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:48.524647 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:48.524652 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:48.525398 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:48.525399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:48.526232 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:48.526235 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:48.526664 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:48.526665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:48.526828 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:48.527591 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:48.527601 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:48.529288 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:48.531221 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:48.531231 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:48.531234 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:48.531577 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:48.531863 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:48.531864 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:48.531864 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:48.532012 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:48.532419 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:48.532427 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:48.532624 (+ 197us) client_negotiation.cc:769] Sending connection context
0808 07:01:48.532756 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 07:01:48.532955 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":152,"threads_started":1}
I20260808 07:01:48.533977 16972 heartbeater.cc:347] Connected to a master server at 127.13.207.62:40627
I20260808 07:01:48.534093 16972 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:48.534305 16972 heartbeater.cc:511] Master 127.13.207.62:40627 requested a full tablet report, sending...
I20260808 07:01:48.534909 16632 ts_manager.cc:194] Registered new tserver with Master: 727529a1f3114a129dbbb84b17e46f9c (127.13.207.2:46709)
I20260808 07:01:48.535490 16632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:40607
WARNING: no policy specified for kudu/127.13.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
I20260808 07:01:48.563025 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40627
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:48.654189 16977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:48.654424 16977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:48.654451 16977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:48.655702 16977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:48.655752 16977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:48.655781 16977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:01:48.657217 16977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:48.657423 16977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:48.657616 16977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:48.663389 16982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:48.663590 16977 server_base.cc:1034] running on GCE node
W20260808 07:01:48.663403 16983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:48.663426 16985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:48.663908 16977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:48.664206 16977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:48.665374 16977 hybrid_clock.cc:648] HybridClock initialized: now 1786172508665344 us; error 67 us; skew 500 ppm
Aug 08 07:01:48 dist-test-slave-8tdl krb5kdc[16587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172508, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:48.668718 16977 init.cc:377] Logged in from keytab as kudu/127.13.207.3@KRBTEST.COM (short username kudu)
I20260808 07:01:48.669346 16977 webserver.cc:460] Webserver started at http://127.13.207.3:38641/ using document root <none> and password file <none>
I20260808 07:01:48.669608 16977 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:48.669682 16977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:48.669837 16977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:48.671059 16977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "01ff1c935df5432b9223738dff61957c"
format_stamp: "Formatted at 2026-08-08 07:01:48 on dist-test-slave-8tdl"
server_key: "f3819028dac8d0618ce7057925b9bd39"
server_key_iv: "9e2efc4e780663065ca084e8c511c7e9"
server_key_version: "encryptionkey@0"
I20260808 07:01:48.671454 16977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "01ff1c935df5432b9223738dff61957c"
format_stamp: "Formatted at 2026-08-08 07:01:48 on dist-test-slave-8tdl"
server_key: "f3819028dac8d0618ce7057925b9bd39"
server_key_iv: "9e2efc4e780663065ca084e8c511c7e9"
server_key_version: "encryptionkey@0"
I20260808 07:01:48.672973 16977 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:01:48.673802 16992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:48.673990 16977 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:48.674103 16977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "01ff1c935df5432b9223738dff61957c"
format_stamp: "Formatted at 2026-08-08 07:01:48 on dist-test-slave-8tdl"
server_key: "f3819028dac8d0618ce7057925b9bd39"
server_key_iv: "9e2efc4e780663065ca084e8c511c7e9"
server_key_version: "encryptionkey@0"
I20260808 07:01:48.674197 16977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:48.685974 16977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:48.688607 16977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:48.688817 16977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:48.689095 16977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:48.689510 16977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:48.689572 16977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:48.689610 16977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:48.689640 16977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:48.696453 16977 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:44665
I20260808 07:01:48.696547 17109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:44665 every 8 connection(s)
I20260808 07:01:48.696906 16977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:01:48.700836 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 16977
I20260808 07:01:48.701015 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:01:48.701210 14140 external_mini_cluster.cc:1428] Setting key d9abba02f0e2fa4ba6cd2f530f939713
Aug 08 07:01:48 dist-test-slave-8tdl krb5kdc[16587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172508, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:48.708226 16675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:48.697844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:53645 (local address 127.13.207.62:40627)
0808 07:01:48.697929 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:48.697931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:48.698164 (+ 233us) server_negotiation.cc:407] Connection header received
0808 07:01:48.698816 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:48.698818 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:48.698896 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:48.698960 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:48.700085 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:48.700612 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:48.702172 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:48.702431 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:48.702542 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:48.705121 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:48.705123 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:48.705123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:48.705139 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:48.706412 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:48.706968 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:48.706970 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:48.706970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:48.707036 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:48.707321 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:48.707323 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:48.707323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:48.707457 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:48.707537 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:48.707851 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 07:01:48.708011 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 07:01:48.708484 17113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:48.697780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:40627 (local address 127.13.207.3:53645)
0808 07:01:48.697979 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:48.698027 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:48.698703 (+ 676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:48.699030 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:48.699034 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:48.699722 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:48.699969 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:48.699974 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:48.700746 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:48.700748 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:48.702088 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:48.702091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:48.702573 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:48.702574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:48.702802 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:48.703384 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:48.703393 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:48.705000 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:48.706510 (+ 1510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:48.706519 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:48.706522 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:48.706883 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:48.707118 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:48.707119 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:48.707119 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:48.707246 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:48.707541 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:48.707548 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:48.707706 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 07:01:48.707813 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 07:01:48.708016 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":116,"threads_started":1}
I20260808 07:01:48.709162 17110 heartbeater.cc:347] Connected to a master server at 127.13.207.62:40627
I20260808 07:01:48.709263 17110 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:48.709523 17110 heartbeater.cc:511] Master 127.13.207.62:40627 requested a full tablet report, sending...
I20260808 07:01:48.710000 16632 ts_manager.cc:194] Registered new tserver with Master: 01ff1c935df5432b9223738dff61957c (127.13.207.3:44665)
I20260808 07:01:48.710362 16632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:53645
I20260808 07:01:48.713976 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 07:01:48.729230 16675 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:48.727935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40402 (local address 127.13.207.62:40627)
0808 07:01:48.728052 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:48.728054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:48.728160 (+ 106us) server_negotiation.cc:407] Connection header received
0808 07:01:48.728259 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:48.728263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:48.728279 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:48.728351 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:48.729115 (+ 764us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40402: BlockingRecv error: recv got EOF from 127.0.0.1:40402 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 07:01:48.730188 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 16699
I20260808 07:01:48.736755 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 16837
I20260808 07:01:48.743603 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 16977
I20260808 07:01:48.750267 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 16600
2026-08-08T07:01:48Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2111 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:01:48 dist-test-slave-8tdl krb5kdc[17123](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:01:48 dist-test-slave-8tdl krb5kdc[17123](info): set up 2 sockets
Aug 08 07:01:48 dist-test-slave-8tdl krb5kdc[17123](info): commencing operation
krb5kdc: starting...
W20260808 07:01:49.873759 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:01:49 dist-test-slave-8tdl krb5kdc[17123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:01:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:01:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:01:50.045509 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:40987
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:40987
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:50.146530 17136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:50.146868 17136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:50.146922 17136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:50.148787 17136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:01:50.148923 17136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:50.148952 17136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:50.148976 17136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:01:50.149008 17136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:01:50.151121 17136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:40987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:40987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:50.151499 17136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:50.151769 17136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:50.158070 17142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:50.158074 17144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:50.158074 17141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:50.158548 17136 server_base.cc:1034] running on GCE node
I20260808 07:01:50.158931 17136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:50.159317 17136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:50.160532 17136 hybrid_clock.cc:648] HybridClock initialized: now 1786172510160509 us; error 47 us; skew 500 ppm
Aug 08 07:01:50 dist-test-slave-8tdl krb5kdc[17123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172510, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:50.170327 17136 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:01:50.171043 17136 webserver.cc:460] Webserver started at http://127.13.207.62:46041/ using document root <none> and password file <none>
I20260808 07:01:50.171340 17136 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:50.171452 17136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:50.171847 17136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:50.173237 17136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "eb15c13bb02f40bcb7c116ecd4cd19ff"
format_stamp: "Formatted at 2026-08-08 07:01:50 on dist-test-slave-8tdl"
server_key: "56083be02dd1ae40616e1e10921a3a43"
server_key_iv: "bd5cc32dc53ce1bb942092ba90f4c2b3"
server_key_version: "encryptionkey@0"
I20260808 07:01:50.173774 17136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "eb15c13bb02f40bcb7c116ecd4cd19ff"
format_stamp: "Formatted at 2026-08-08 07:01:50 on dist-test-slave-8tdl"
server_key: "56083be02dd1ae40616e1e10921a3a43"
server_key_iv: "bd5cc32dc53ce1bb942092ba90f4c2b3"
server_key_version: "encryptionkey@0"
I20260808 07:01:50.175676 17136 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 07:01:50.176889 17151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:50.177254 17136 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:50.177419 17136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "eb15c13bb02f40bcb7c116ecd4cd19ff"
format_stamp: "Formatted at 2026-08-08 07:01:50 on dist-test-slave-8tdl"
server_key: "56083be02dd1ae40616e1e10921a3a43"
server_key_iv: "bd5cc32dc53ce1bb942092ba90f4c2b3"
server_key_version: "encryptionkey@0"
I20260808 07:01:50.177688 17136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:50.191406 17136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:50.195423 17136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:50.195724 17136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:50.200954 17136 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:40987
I20260808 07:01:50.201406 17206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:40987 every 8 connection(s)
I20260808 07:01:50.201419 17136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:01:50.202662 17207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:50.204089 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 17136
I20260808 07:01:50.204200 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:01:50.204442 14140 external_mini_cluster.cc:1428] Setting key 7c2211ca07fb846a4b44343ab8301069
I20260808 07:01:50.205780 17207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb15c13bb02f40bcb7c116ecd4cd19ff: Bootstrap starting.
I20260808 07:01:50.206542 17207 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb15c13bb02f40bcb7c116ecd4cd19ff: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:50.206907 17207 log.cc:824] T 00000000000000000000000000000000 P eb15c13bb02f40bcb7c116ecd4cd19ff: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:50.207808 17207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb15c13bb02f40bcb7c116ecd4cd19ff: No bootstrap required, opened a new log
I20260808 07:01:50.210282 17207 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb15c13bb02f40bcb7c116ecd4cd19ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb15c13bb02f40bcb7c116ecd4cd19ff" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40987 } }
I20260808 07:01:50.210479 17207 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb15c13bb02f40bcb7c116ecd4cd19ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:50.210508 17207 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb15c13bb02f40bcb7c116ecd4cd19ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb15c13bb02f40bcb7c116ecd4cd19ff, State: Initialized, Role: FOLLOWER
I20260808 07:01:50.210654 17207 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb15c13bb02f40bcb7c116ecd4cd19ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb15c13bb02f40bcb7c116ecd4cd19ff" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40987 } }
I20260808 07:01:50.210757 17207 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb15c13bb02f40bcb7c116ecd4cd19ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:01:50.210788 17207 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb15c13bb02f40bcb7c116ecd4cd19ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:01:50.210829 17207 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb15c13bb02f40bcb7c116ecd4cd19ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:50.211562 17207 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb15c13bb02f40bcb7c116ecd4cd19ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb15c13bb02f40bcb7c116ecd4cd19ff" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40987 } }
I20260808 07:01:50.211663 17207 leader_election.cc:302] T 00000000000000000000000000000000 P eb15c13bb02f40bcb7c116ecd4cd19ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb15c13bb02f40bcb7c116ecd4cd19ff; no voters:
I20260808 07:01:50.211925 17207 leader_election.cc:288] T 00000000000000000000000000000000 P eb15c13bb02f40bcb7c116ecd4cd19ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:01:50.211959 17212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb15c13bb02f40bcb7c116ecd4cd19ff [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:50.212101 17212 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb15c13bb02f40bcb7c116ecd4cd19ff [term 1 LEADER]: Becoming Leader. State: Replica: eb15c13bb02f40bcb7c116ecd4cd19ff, State: Running, Role: LEADER
I20260808 07:01:50.212191 17212 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb15c13bb02f40bcb7c116ecd4cd19ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb15c13bb02f40bcb7c116ecd4cd19ff" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40987 } }
I20260808 07:01:50.212239 17207 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb15c13bb02f40bcb7c116ecd4cd19ff [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:01:50.212683 17212 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb15c13bb02f40bcb7c116ecd4cd19ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb15c13bb02f40bcb7c116ecd4cd19ff. Latest consensus state: current_term: 1 leader_uuid: "eb15c13bb02f40bcb7c116ecd4cd19ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb15c13bb02f40bcb7c116ecd4cd19ff" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40987 } } }
I20260808 07:01:50.212797 17212 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb15c13bb02f40bcb7c116ecd4cd19ff [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:50.212678 17214 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb15c13bb02f40bcb7c116ecd4cd19ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb15c13bb02f40bcb7c116ecd4cd19ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb15c13bb02f40bcb7c116ecd4cd19ff" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40987 } } }
Aug 08 07:01:50 dist-test-slave-8tdl krb5kdc[17123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:50.213203 17214 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb15c13bb02f40bcb7c116ecd4cd19ff [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:50.214123 17217 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:01:50.214771 17217 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:01:50.216670 17217 catalog_manager.cc:1347] Generated new cluster ID: 6ebdfe58285640bcafa87b8477e8c22b
I20260808 07:01:50.216738 17217 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:01:50.218470 17210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:50.204995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55870 (local address 127.13.207.62:40987)
0808 07:01:50.205233 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:50.205244 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:50.205279 (+ 35us) server_negotiation.cc:407] Connection header received
0808 07:01:50.205867 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:50.205877 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:50.205943 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:50.206111 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:50.207068 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:50.208400 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:50.209346 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:50.210112 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:50.210239 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:50.213946 (+ 3707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:50.213951 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:50.213952 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:50.213969 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:50.216538 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:50.217068 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:50.217072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:50.217073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:50.217191 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:50.217460 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:50.217462 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:50.217462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:50.217744 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:50.217966 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:50.218044 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 07:01:50.218211 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":84,"threads_started":1}
I20260808 07:01:50.223923 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:01:50.234462 17217 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:01:50.235291 17217 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:01:50.242875 17210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:50.238861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55876 (local address 127.13.207.62:40987)
0808 07:01:50.238916 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:50.238919 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:50.239163 (+ 244us) server_negotiation.cc:407] Connection header received
0808 07:01:50.239252 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:50.239253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:50.239265 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:50.239316 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:50.239763 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:50.240319 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:50.241138 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:50.241379 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:50.241463 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:50.242313 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:50.242314 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:50.242314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:50.242539 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:50.242557 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:50.242564 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:50.242639 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:50.242742 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 07:01:50.242776 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:01:50.250267 17217 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb15c13bb02f40bcb7c116ecd4cd19ff: Generated new TSK 0
I20260808 07:01:50.250561 17217 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:01:50.255910 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 17136
2026-08-08T07:01:50Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1503 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:01:50 dist-test-slave-8tdl krb5kdc[17240](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:01:50 dist-test-slave-8tdl krb5kdc[17240](info): set up 2 sockets
Aug 08 07:01:50 dist-test-slave-8tdl krb5kdc[17240](info): commencing operation
krb5kdc: starting...
W20260808 07:01:51.411238 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:01:51 dist-test-slave-8tdl krb5kdc[17240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:01:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:01:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:01:51.559602 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:43841
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:43841
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:51.653529 17253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:51.653831 17253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:51.653888 17253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:51.655664 17253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:01:51.655797 17253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:51.655840 17253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:51.655880 17253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:01:51.655907 17253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:01:51.658133 17253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:43841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:43841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:51.658515 17253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:51.658833 17253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:51.664664 17260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:51.664841 17253 server_base.cc:1034] running on GCE node
W20260808 07:01:51.664675 17262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:51.664693 17259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:51.665349 17253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:51.665707 17253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:51.666898 17253 hybrid_clock.cc:648] HybridClock initialized: now 1786172511666844 us; error 70 us; skew 500 ppm
Aug 08 07:01:51 dist-test-slave-8tdl krb5kdc[17240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172511, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:51.670114 17253 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:01:51.670749 17253 webserver.cc:460] Webserver started at http://127.13.207.62:46865/ using document root <none> and password file <none>
I20260808 07:01:51.671023 17253 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:51.671100 17253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:51.671267 17253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:51.672582 17253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "aea9cab0eb0f4f4db9e1fe22e3221714"
format_stamp: "Formatted at 2026-08-08 07:01:51 on dist-test-slave-8tdl"
server_key: "61466610e65691e3417638359ba32e89"
server_key_iv: "adba9e6de83a1fef66795871e859faaa"
server_key_version: "encryptionkey@0"
I20260808 07:01:51.672986 17253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "aea9cab0eb0f4f4db9e1fe22e3221714"
format_stamp: "Formatted at 2026-08-08 07:01:51 on dist-test-slave-8tdl"
server_key: "61466610e65691e3417638359ba32e89"
server_key_iv: "adba9e6de83a1fef66795871e859faaa"
server_key_version: "encryptionkey@0"
I20260808 07:01:51.674600 17253 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:01:51.675570 17269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:51.675783 17253 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:51.675899 17253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "aea9cab0eb0f4f4db9e1fe22e3221714"
format_stamp: "Formatted at 2026-08-08 07:01:51 on dist-test-slave-8tdl"
server_key: "61466610e65691e3417638359ba32e89"
server_key_iv: "adba9e6de83a1fef66795871e859faaa"
server_key_version: "encryptionkey@0"
I20260808 07:01:51.676007 17253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:51.693428 17253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:51.696323 17253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:51.696578 17253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:51.701679 17253 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:43841
I20260808 07:01:51.701718 17324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:43841 every 8 connection(s)
I20260808 07:01:51.702114 17253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:01:51.703502 17325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:51.706084 17325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aea9cab0eb0f4f4db9e1fe22e3221714: Bootstrap starting.
I20260808 07:01:51.706779 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 17253
I20260808 07:01:51.706899 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:01:51.706955 17325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aea9cab0eb0f4f4db9e1fe22e3221714: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:51.707194 14140 external_mini_cluster.cc:1428] Setting key 4b6c4c3acc7cbbc96b5c121fb18904a3
I20260808 07:01:51.707612 17325 log.cc:824] T 00000000000000000000000000000000 P aea9cab0eb0f4f4db9e1fe22e3221714: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:51.708777 17325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aea9cab0eb0f4f4db9e1fe22e3221714: No bootstrap required, opened a new log
I20260808 07:01:51.711580 17325 raft_consensus.cc:348] T 00000000000000000000000000000000 P aea9cab0eb0f4f4db9e1fe22e3221714 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aea9cab0eb0f4f4db9e1fe22e3221714" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 43841 } }
I20260808 07:01:51.711827 17325 raft_consensus.cc:374] T 00000000000000000000000000000000 P aea9cab0eb0f4f4db9e1fe22e3221714 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:51.711874 17325 raft_consensus.cc:729] T 00000000000000000000000000000000 P aea9cab0eb0f4f4db9e1fe22e3221714 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aea9cab0eb0f4f4db9e1fe22e3221714, State: Initialized, Role: FOLLOWER
I20260808 07:01:51.712021 17325 consensus_queue.cc:260] T 00000000000000000000000000000000 P aea9cab0eb0f4f4db9e1fe22e3221714 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aea9cab0eb0f4f4db9e1fe22e3221714" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 43841 } }
I20260808 07:01:51.712124 17325 raft_consensus.cc:388] T 00000000000000000000000000000000 P aea9cab0eb0f4f4db9e1fe22e3221714 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:01:51.712158 17325 raft_consensus.cc:482] T 00000000000000000000000000000000 P aea9cab0eb0f4f4db9e1fe22e3221714 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:01:51.712212 17325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aea9cab0eb0f4f4db9e1fe22e3221714 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:51.713052 17325 raft_consensus.cc:504] T 00000000000000000000000000000000 P aea9cab0eb0f4f4db9e1fe22e3221714 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aea9cab0eb0f4f4db9e1fe22e3221714" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 43841 } }
I20260808 07:01:51.713155 17325 leader_election.cc:302] T 00000000000000000000000000000000 P aea9cab0eb0f4f4db9e1fe22e3221714 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aea9cab0eb0f4f4db9e1fe22e3221714; no voters:
I20260808 07:01:51.713403 17325 leader_election.cc:288] T 00000000000000000000000000000000 P aea9cab0eb0f4f4db9e1fe22e3221714 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:01:51.713574 17330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aea9cab0eb0f4f4db9e1fe22e3221714 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:51.713789 17330 raft_consensus.cc:686] T 00000000000000000000000000000000 P aea9cab0eb0f4f4db9e1fe22e3221714 [term 1 LEADER]: Becoming Leader. State: Replica: aea9cab0eb0f4f4db9e1fe22e3221714, State: Running, Role: LEADER
I20260808 07:01:51.713972 17330 consensus_queue.cc:237] T 00000000000000000000000000000000 P aea9cab0eb0f4f4db9e1fe22e3221714 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aea9cab0eb0f4f4db9e1fe22e3221714" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 43841 } }
I20260808 07:01:51.714169 17325 sys_catalog.cc:564] T 00000000000000000000000000000000 P aea9cab0eb0f4f4db9e1fe22e3221714 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:01:51.714408 17332 sys_catalog.cc:455] T 00000000000000000000000000000000 P aea9cab0eb0f4f4db9e1fe22e3221714 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aea9cab0eb0f4f4db9e1fe22e3221714. Latest consensus state: current_term: 1 leader_uuid: "aea9cab0eb0f4f4db9e1fe22e3221714" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aea9cab0eb0f4f4db9e1fe22e3221714" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 43841 } } }
I20260808 07:01:51.714511 17332 sys_catalog.cc:458] T 00000000000000000000000000000000 P aea9cab0eb0f4f4db9e1fe22e3221714 [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:51.714706 17331 sys_catalog.cc:455] T 00000000000000000000000000000000 P aea9cab0eb0f4f4db9e1fe22e3221714 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aea9cab0eb0f4f4db9e1fe22e3221714" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aea9cab0eb0f4f4db9e1fe22e3221714" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 43841 } } }
I20260808 07:01:51.714784 17331 sys_catalog.cc:458] T 00000000000000000000000000000000 P aea9cab0eb0f4f4db9e1fe22e3221714 [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:51.714946 17334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:01:51.715716 17334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 07:01:51 dist-test-slave-8tdl krb5kdc[17240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:51.718266 17334 catalog_manager.cc:1347] Generated new cluster ID: ac5cb01abdd14166a1c1a464611b6c42
I20260808 07:01:51.718379 17334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:01:51.725657 17329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:51.708060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60142 (local address 127.13.207.62:43841)
0808 07:01:51.708237 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:51.708246 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:51.708474 (+ 228us) server_negotiation.cc:407] Connection header received
0808 07:01:51.709154 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:51.709162 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:51.709217 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:51.709345 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:51.710320 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:51.711449 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:51.712685 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:51.713313 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:51.713440 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:51.716899 (+ 3459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:51.716905 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:51.716906 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:51.716921 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:51.719162 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:51.719670 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:51.719674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:51.719674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:51.719785 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:51.720028 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:51.720029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:51.720029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:51.720339 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:51.724841 (+ 4502us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:51.725008 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 07:01:51.725293 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260808 07:01:51.733541 17334 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:01:51.734333 17334 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:01:51.740115 17334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aea9cab0eb0f4f4db9e1fe22e3221714: Generated new TSK 0
I20260808 07:01:51.740410 17334 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
I20260808 07:01:51.780637 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:43841
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:51.877815 17351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:51.878114 17351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:51.878163 17351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:51.879643 17351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:51.879727 17351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:51.879776 17351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:01:51.881496 17351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:43841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:51.881824 17351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:51.882107 17351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:51.888489 17360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:51.888449 17357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:51.888801 17351 server_base.cc:1034] running on GCE node
W20260808 07:01:51.888808 17358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:51.889132 17351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:51.889456 17351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:51.890657 17351 hybrid_clock.cc:648] HybridClock initialized: now 1786172511890631 us; error 42 us; skew 500 ppm
Aug 08 07:01:51 dist-test-slave-8tdl krb5kdc[17240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172511, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:51.893998 17351 init.cc:377] Logged in from keytab as kudu/127.13.207.1@KRBTEST.COM (short username kudu)
I20260808 07:01:51.895071 17351 webserver.cc:460] Webserver started at http://127.13.207.1:35057/ using document root <none> and password file <none>
I20260808 07:01:51.895356 17351 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:51.895438 17351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:51.895614 17351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:51.896811 17351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "b10ea8e4be3e4e718042b154cedb9866"
format_stamp: "Formatted at 2026-08-08 07:01:51 on dist-test-slave-8tdl"
server_key: "a5e62f998d4c26866deda60d01ea3d09"
server_key_iv: "c05ec1028d29e0112baef2fa7ad476ff"
server_key_version: "encryptionkey@0"
I20260808 07:01:51.897235 17351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "b10ea8e4be3e4e718042b154cedb9866"
format_stamp: "Formatted at 2026-08-08 07:01:51 on dist-test-slave-8tdl"
server_key: "a5e62f998d4c26866deda60d01ea3d09"
server_key_iv: "c05ec1028d29e0112baef2fa7ad476ff"
server_key_version: "encryptionkey@0"
I20260808 07:01:51.898907 17351 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260808 07:01:51.899992 17368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:51.900298 17351 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:51.900472 17351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "b10ea8e4be3e4e718042b154cedb9866"
format_stamp: "Formatted at 2026-08-08 07:01:51 on dist-test-slave-8tdl"
server_key: "a5e62f998d4c26866deda60d01ea3d09"
server_key_iv: "c05ec1028d29e0112baef2fa7ad476ff"
server_key_version: "encryptionkey@0"
I20260808 07:01:51.900589 17351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:51.934430 17351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:51.937515 17351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:51.937747 17351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:51.938046 17351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:51.938411 17351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:51.938486 17351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:51.938531 17351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:51.938561 17351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:51.945221 17351 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:36803
I20260808 07:01:51.945335 17486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:36803 every 8 connection(s)
I20260808 07:01:51.945708 17351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:01:51.948361 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 17351
I20260808 07:01:51.948482 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:01:51.948691 14140 external_mini_cluster.cc:1428] Setting key 8fcc05b3a7660cac47c78c272bc01723
Aug 08 07:01:51 dist-test-slave-8tdl krb5kdc[17240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172511, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:51.959939 17329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:51.946656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:60489 (local address 127.13.207.62:43841)
0808 07:01:51.946748 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:51.946751 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:51.946981 (+ 230us) server_negotiation.cc:407] Connection header received
0808 07:01:51.947635 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:51.947637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:51.947652 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:51.947739 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:51.949480 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:51.950021 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:51.951972 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:51.952408 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:51.952514 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:51.955864 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:51.955866 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:51.955866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:51.955876 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:51.957741 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:51.958514 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:51.958515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:51.958515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:51.958591 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:51.959038 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:51.959039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:51.959039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:51.959191 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:51.959253 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:51.959754 (+ 501us) server_negotiation.cc:299] Negotiation successful
0808 07:01:51.959852 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
WARNING: no policy specified for kudu/127.13.207.2@KRBTEST.COM; defaulting to no policy
I20260808 07:01:51.960220 17490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:51.946567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:43841 (local address 127.13.207.1:60489)
0808 07:01:51.946764 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:51.946806 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:51.947414 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:51.947888 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:51.947892 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:51.948811 (+ 919us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:51.949231 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:51.949245 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:51.950319 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:51.950320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:51.951826 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:51.951830 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:51.952564 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:51.952566 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:51.952789 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:51.953498 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:51.953509 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:51.955711 (+ 2202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:51.957874 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:51.957891 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:51.957894 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:51.958374 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:51.958711 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:51.958712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:51.958713 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:51.958898 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:51.959276 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:51.959287 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:51.959535 (+ 248us) client_negotiation.cc:769] Sending connection context
0808 07:01:51.959680 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 07:01:51.959901 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":77,"threads_started":1}
I20260808 07:01:51.961977 17487 heartbeater.cc:347] Connected to a master server at 127.13.207.62:43841
I20260808 07:01:51.962495 17487 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:51.962853 17487 heartbeater.cc:511] Master 127.13.207.62:43841 requested a full tablet report, sending...
I20260808 07:01:51.963737 17287 ts_manager.cc:194] Registered new tserver with Master: b10ea8e4be3e4e718042b154cedb9866 (127.13.207.1:36803)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.2@KRBTEST.COM" created.
I20260808 07:01:51.964690 17287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:60489
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
I20260808 07:01:51.983245 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:43841
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:52.077780 17493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:52.078027 17493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:52.078054 17493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:52.079528 17493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:52.079629 17493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:52.079685 17493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:01:52.081410 17493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:43841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:52.081735 17493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:52.082043 17493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:52.088013 17498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:52.088002 17501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:52.088002 17499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:52.088581 17493 server_base.cc:1034] running on GCE node
I20260808 07:01:52.088836 17493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:52.089118 17493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:52.090297 17493 hybrid_clock.cc:648] HybridClock initialized: now 1786172512090256 us; error 60 us; skew 500 ppm
Aug 08 07:01:52 dist-test-slave-8tdl krb5kdc[17240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172512, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:52.093545 17493 init.cc:377] Logged in from keytab as kudu/127.13.207.2@KRBTEST.COM (short username kudu)
I20260808 07:01:52.094075 17493 webserver.cc:460] Webserver started at http://127.13.207.2:36975/ using document root <none> and password file <none>
I20260808 07:01:52.094348 17493 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:52.094398 17493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:52.094537 17493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:52.095768 17493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "c72a47f505a446a889cb47c113566521"
format_stamp: "Formatted at 2026-08-08 07:01:52 on dist-test-slave-8tdl"
server_key: "9c29d943969b78ed3d96595930a056d8"
server_key_iv: "942424e66597d6a4f6abc4f513eb9f42"
server_key_version: "encryptionkey@0"
I20260808 07:01:52.096158 17493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "c72a47f505a446a889cb47c113566521"
format_stamp: "Formatted at 2026-08-08 07:01:52 on dist-test-slave-8tdl"
server_key: "9c29d943969b78ed3d96595930a056d8"
server_key_iv: "942424e66597d6a4f6abc4f513eb9f42"
server_key_version: "encryptionkey@0"
I20260808 07:01:52.097720 17493 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:01:52.098951 17508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:52.099393 17493 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:52.099488 17493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "c72a47f505a446a889cb47c113566521"
format_stamp: "Formatted at 2026-08-08 07:01:52 on dist-test-slave-8tdl"
server_key: "9c29d943969b78ed3d96595930a056d8"
server_key_iv: "942424e66597d6a4f6abc4f513eb9f42"
server_key_version: "encryptionkey@0"
I20260808 07:01:52.099622 17493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:52.132357 17493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:52.135529 17493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:52.135710 17493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:52.136029 17493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:52.136405 17493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:52.136466 17493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:52.136502 17493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:52.136526 17493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:52.143599 17493 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:46251
I20260808 07:01:52.143651 17624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:46251 every 8 connection(s)
I20260808 07:01:52.144044 17493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:01:52.151235 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 17493
I20260808 07:01:52.151391 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:01:52.151618 14140 external_mini_cluster.cc:1428] Setting key b603f369bcb152c717bc73731a8a7cf2
Aug 08 07:01:52 dist-test-slave-8tdl krb5kdc[17240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172512, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:52.157155 17329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:52.145114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:58583 (local address 127.13.207.62:43841)
0808 07:01:52.145222 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:52.145225 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:52.145430 (+ 205us) server_negotiation.cc:407] Connection header received
0808 07:01:52.146039 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:52.146041 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:52.146054 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:52.146105 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:52.147333 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:52.147876 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:52.148956 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:52.149173 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:52.149270 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:52.152962 (+ 3692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:52.152972 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:52.152973 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:52.152992 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:52.155077 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:52.155755 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:52.155757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:52.155757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:52.155828 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:52.156311 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:52.156312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:52.156312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:52.156450 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:52.156521 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:52.156993 (+ 472us) server_negotiation.cc:299] Negotiation successful
0808 07:01:52.157080 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 07:01:52.157407 17627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:52.144966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:43841 (local address 127.13.207.2:58583)
0808 07:01:52.145264 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:52.145334 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:52.145919 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:52.146191 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:52.146196 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:52.146933 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:52.147218 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:52.147224 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:52.147992 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:52.147993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:52.148848 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:52.148850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:52.149287 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:52.149288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:52.149426 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:52.150186 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:52.150196 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:52.152774 (+ 2578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:52.155209 (+ 2435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:52.155223 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:52.155227 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:52.155644 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:52.155912 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:52.155913 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:52.155913 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:52.156183 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:52.156538 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:52.156547 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:52.156750 (+ 203us) client_negotiation.cc:769] Sending connection context
0808 07:01:52.156877 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 07:01:52.157102 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":129,"threads_started":1}
I20260808 07:01:52.158030 17625 heartbeater.cc:347] Connected to a master server at 127.13.207.62:43841
I20260808 07:01:52.158177 17625 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:52.158396 17625 heartbeater.cc:511] Master 127.13.207.62:43841 requested a full tablet report, sending...
I20260808 07:01:52.158905 17287 ts_manager.cc:194] Registered new tserver with Master: c72a47f505a446a889cb47c113566521 (127.13.207.2:46251)
I20260808 07:01:52.159355 17287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:58583
WARNING: no policy specified for kudu/127.13.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
I20260808 07:01:52.187477 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:43841
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:52.278141 17630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:52.278425 17630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:52.278469 17630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:52.279932 17630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:52.280030 17630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:52.280082 17630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:01:52.281836 17630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:43841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:52.282176 17630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:52.282445 17630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:52.288264 17639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:52.288239 17635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:52.288461 17630 server_base.cc:1034] running on GCE node
W20260808 07:01:52.288220 17636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:52.288805 17630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:52.289114 17630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:52.290351 17630 hybrid_clock.cc:648] HybridClock initialized: now 1786172512290325 us; error 45 us; skew 500 ppm
Aug 08 07:01:52 dist-test-slave-8tdl krb5kdc[17240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172512, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:52.293717 17630 init.cc:377] Logged in from keytab as kudu/127.13.207.3@KRBTEST.COM (short username kudu)
I20260808 07:01:52.294350 17630 webserver.cc:460] Webserver started at http://127.13.207.3:45343/ using document root <none> and password file <none>
I20260808 07:01:52.294639 17630 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:52.294750 17630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:52.294948 17630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:52.296114 17630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "bb42d4214c0a418aa18c7ad11ab59b72"
format_stamp: "Formatted at 2026-08-08 07:01:52 on dist-test-slave-8tdl"
server_key: "7bfd32a4fb597b7eb664c1e0fef35956"
server_key_iv: "eca0c95b8b87f42fd4e66cc2a7ddb382"
server_key_version: "encryptionkey@0"
I20260808 07:01:52.296532 17630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "bb42d4214c0a418aa18c7ad11ab59b72"
format_stamp: "Formatted at 2026-08-08 07:01:52 on dist-test-slave-8tdl"
server_key: "7bfd32a4fb597b7eb664c1e0fef35956"
server_key_iv: "eca0c95b8b87f42fd4e66cc2a7ddb382"
server_key_version: "encryptionkey@0"
I20260808 07:01:52.298233 17630 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:01:52.299317 17646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:52.299574 17630 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:01:52.299713 17630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "bb42d4214c0a418aa18c7ad11ab59b72"
format_stamp: "Formatted at 2026-08-08 07:01:52 on dist-test-slave-8tdl"
server_key: "7bfd32a4fb597b7eb664c1e0fef35956"
server_key_iv: "eca0c95b8b87f42fd4e66cc2a7ddb382"
server_key_version: "encryptionkey@0"
I20260808 07:01:52.299804 17630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:52.315229 17630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:52.318089 17630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:52.318302 17630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:52.318626 17630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:52.319079 17630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:52.319128 17630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:52.319167 17630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:52.319190 17630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:52.325754 17630 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:43889
I20260808 07:01:52.325799 17763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:43889 every 8 connection(s)
I20260808 07:01:52.326213 17630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:01:52 dist-test-slave-8tdl krb5kdc[17240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172512, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:52.334555 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 17630
I20260808 07:01:52.334654 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:01:52.334884 14140 external_mini_cluster.cc:1428] Setting key 51d7188ed17351549c4eebcad4d9737c
I20260808 07:01:52.337940 17329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:52.327277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:50417 (local address 127.13.207.62:43841)
0808 07:01:52.327327 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:52.327329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:52.327631 (+ 302us) server_negotiation.cc:407] Connection header received
0808 07:01:52.328251 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:52.328253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:52.328267 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:52.328333 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:52.329517 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:52.330105 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:52.331390 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:52.331632 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:52.331735 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:52.334201 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:52.334203 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:52.334203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:52.334212 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:52.336346 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:52.336805 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:52.336806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:52.336806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:52.336922 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:52.337237 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:52.337238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:52.337239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:52.337382 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:52.337464 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:52.337789 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 07:01:52.337868 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:01:52.338187 17766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:52.327237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:43841 (local address 127.13.207.3:50417)
0808 07:01:52.327465 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:52.327510 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:52.328107 (+ 597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:52.328462 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:52.328466 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:52.329144 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:52.329369 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:52.329385 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:52.330222 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:52.330223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:52.331272 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:52.331275 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:52.331749 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:52.331750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:52.331898 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:52.332525 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:52.332534 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:52.334079 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:52.336423 (+ 2344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:52.336432 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:52.336434 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:52.336711 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:52.337003 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:52.337004 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:52.337004 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:52.337126 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:52.337502 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:52.337509 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:52.337652 (+ 143us) client_negotiation.cc:769] Sending connection context
0808 07:01:52.337752 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 07:01:52.337950 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":97,"threads_started":1}
I20260808 07:01:52.338794 17764 heartbeater.cc:347] Connected to a master server at 127.13.207.62:43841
I20260808 07:01:52.338903 17764 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:52.339062 17764 heartbeater.cc:511] Master 127.13.207.62:43841 requested a full tablet report, sending...
I20260808 07:01:52.339491 17287 ts_manager.cc:194] Registered new tserver with Master: bb42d4214c0a418aa18c7ad11ab59b72 (127.13.207.3:43889)
I20260808 07:01:52.339915 17287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:50417
I20260808 07:01:52.347299 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 07:01:52 dist-test-slave-8tdl krb5kdc[17240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172512, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 07:01:52 dist-test-slave-8tdl krb5kdc[17240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172512, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:52.379777 17329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:52.368980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60154 (local address 127.13.207.62:43841)
0808 07:01:52.369099 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:52.369101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:52.369114 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:01:52.369277 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:52.369278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:52.369290 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:52.369363 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:52.370186 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:52.371323 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:52.372658 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:52.372870 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:52.372939 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:52.376876 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:52.376883 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:52.376883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:52.376892 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:52.378348 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:52.378848 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:52.378850 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:52.378850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:52.378916 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:52.379142 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:52.379143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:52.379143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:52.379322 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:52.379487 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:52.379568 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 07:01:52.379664 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
W20260808 07:01:52.380162 17287 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:60154
I20260808 07:01:52.380961 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 17351
I20260808 07:01:52.387735 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 17493
I20260808 07:01:52.396299 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 17630
I20260808 07:01:52.403257 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 17253
2026-08-08T07:01:52Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2149 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:01:52 dist-test-slave-8tdl krb5kdc[17776](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:01:52 dist-test-slave-8tdl krb5kdc[17776](info): set up 2 sockets
Aug 08 07:01:52 dist-test-slave-8tdl krb5kdc[17776](info): commencing operation
krb5kdc: starting...
W20260808 07:01:53.527642 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:01:53 dist-test-slave-8tdl krb5kdc[17776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:01:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:01:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:01:53.679185 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:37033
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:37033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:53.774495 17789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:53.774889 17789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:53.774936 17789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:53.776284 17789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:01:53.776364 17789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:53.776388 17789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:53.776409 17789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:01:53.776437 17789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:01:53.778299 17789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:37033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:37033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:53.778616 17789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:53.778905 17789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:53.784373 17794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:53.784394 17797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:53.784653 17789 server_base.cc:1034] running on GCE node
W20260808 07:01:53.784394 17795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:53.785072 17789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:53.785369 17789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:53.786556 17789 hybrid_clock.cc:648] HybridClock initialized: now 1786172513786537 us; error 32 us; skew 500 ppm
Aug 08 07:01:53 dist-test-slave-8tdl krb5kdc[17776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172513, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:53.789861 17789 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:01:53.790448 17789 webserver.cc:460] Webserver started at http://127.13.207.62:34493/ using document root <none> and password file <none>
I20260808 07:01:53.790771 17789 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:53.790825 17789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:53.790956 17789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:53.792117 17789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "fdaaf0d7e85047c383f3eb90754ebd51"
format_stamp: "Formatted at 2026-08-08 07:01:53 on dist-test-slave-8tdl"
server_key: "a68a20a89faedc0e1cb29a3c5d6e46e9"
server_key_iv: "0ff554ced386912d7d7e59fd62f7f777"
server_key_version: "encryptionkey@0"
I20260808 07:01:53.792512 17789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "fdaaf0d7e85047c383f3eb90754ebd51"
format_stamp: "Formatted at 2026-08-08 07:01:53 on dist-test-slave-8tdl"
server_key: "a68a20a89faedc0e1cb29a3c5d6e46e9"
server_key_iv: "0ff554ced386912d7d7e59fd62f7f777"
server_key_version: "encryptionkey@0"
I20260808 07:01:53.794099 17789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:01:53.795181 17804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:53.795500 17789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:53.795631 17789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "fdaaf0d7e85047c383f3eb90754ebd51"
format_stamp: "Formatted at 2026-08-08 07:01:53 on dist-test-slave-8tdl"
server_key: "a68a20a89faedc0e1cb29a3c5d6e46e9"
server_key_iv: "0ff554ced386912d7d7e59fd62f7f777"
server_key_version: "encryptionkey@0"
I20260808 07:01:53.795712 17789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:53.814333 17789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:53.817217 17789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:53.817366 17789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:53.822203 17789 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:37033
I20260808 07:01:53.822256 17859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:37033 every 8 connection(s)
I20260808 07:01:53.823022 17789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:01:53.823786 17860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:53.825789 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 17789
I20260808 07:01:53.825901 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:01:53.826083 14140 external_mini_cluster.cc:1428] Setting key 8ca00a82b584f6243698b01677446cc3
I20260808 07:01:53.826642 17860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51: Bootstrap starting.
I20260808 07:01:53.827374 17860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:53.827706 17860 log.cc:824] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:53.828553 17860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51: No bootstrap required, opened a new log
I20260808 07:01:53.831887 17860 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdaaf0d7e85047c383f3eb90754ebd51" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 37033 } }
I20260808 07:01:53.832074 17860 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:53.832105 17860 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdaaf0d7e85047c383f3eb90754ebd51, State: Initialized, Role: FOLLOWER
I20260808 07:01:53.832265 17860 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdaaf0d7e85047c383f3eb90754ebd51" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 37033 } }
I20260808 07:01:53.832374 17860 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:01:53.832419 17860 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:01:53.832464 17860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:53.833367 17860 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdaaf0d7e85047c383f3eb90754ebd51" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 37033 } }
I20260808 07:01:53.833531 17860 leader_election.cc:302] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdaaf0d7e85047c383f3eb90754ebd51; no voters:
I20260808 07:01:53.833765 17860 leader_election.cc:288] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:01:53.833793 17865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:53.834118 17860 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:01:53.834105 17865 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51 [term 1 LEADER]: Becoming Leader. State: Replica: fdaaf0d7e85047c383f3eb90754ebd51, State: Running, Role: LEADER
I20260808 07:01:53.834662 17865 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdaaf0d7e85047c383f3eb90754ebd51" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 37033 } }
Aug 08 07:01:53 dist-test-slave-8tdl krb5kdc[17776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:53.835114 17865 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdaaf0d7e85047c383f3eb90754ebd51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdaaf0d7e85047c383f3eb90754ebd51" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 37033 } } }
I20260808 07:01:53.835213 17865 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51 [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:53.835330 17865 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdaaf0d7e85047c383f3eb90754ebd51. Latest consensus state: current_term: 1 leader_uuid: "fdaaf0d7e85047c383f3eb90754ebd51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdaaf0d7e85047c383f3eb90754ebd51" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 37033 } } }
I20260808 07:01:53.835386 17865 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51 [sys.catalog]: This master's current role is: LEADER
W20260808 07:01:53.836544 17881 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:01:53.836601 17881 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:01:53.836651 17874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:01:53.836849 17874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:01:53.838568 17874 catalog_manager.cc:1347] Generated new cluster ID: 416064e322a544aab0694963b3b2406d
I20260808 07:01:53.838625 17874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:01:53.841279 17863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:53.826738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44692 (local address 127.13.207.62:37033)
0808 07:01:53.826963 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:53.826974 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:53.827013 (+ 39us) server_negotiation.cc:407] Connection header received
0808 07:01:53.827568 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:53.827577 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:53.827643 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:53.827773 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:53.828628 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:53.829751 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:53.831669 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:53.832215 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:53.832361 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:53.835687 (+ 3326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:53.835693 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:53.835693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:53.835709 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:53.839359 (+ 3650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:53.839898 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:53.839901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:53.839902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:53.840022 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:53.840247 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:53.840248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:53.840248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:53.840579 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:53.840748 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:53.840879 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 07:01:53.841038 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":127,"threads_started":1}
I20260808 07:01:53.858153 17874 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:01:53.858846 17874 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:01:53.870054 17874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdaaf0d7e85047c383f3eb90754ebd51: Generated new TSK 0
I20260808 07:01:53.870466 17874 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
I20260808 07:01:53.908057 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:37033
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:53.999259 17887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:53.999501 17887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:53.999527 17887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:54.001155 17887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:54.001236 17887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:54.001271 17887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:01:54.003419 17887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:37033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:54.003739 17887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:54.004035 17887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:01:54.011024 17887 server_base.cc:1034] running on GCE node
W20260808 07:01:54.011008 17896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:54.011004 17894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:54.011970 17893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:54.012333 17887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:54.012682 17887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:54.013971 17887 hybrid_clock.cc:648] HybridClock initialized: now 1786172514013959 us; error 36 us; skew 500 ppm
Aug 08 07:01:54 dist-test-slave-8tdl krb5kdc[17776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172514, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:54.019577 17887 init.cc:377] Logged in from keytab as kudu/127.13.207.1@KRBTEST.COM (short username kudu)
I20260808 07:01:54.020226 17887 webserver.cc:460] Webserver started at http://127.13.207.1:33547/ using document root <none> and password file <none>
I20260808 07:01:54.020495 17887 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:54.020570 17887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:54.020733 17887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:54.021886 17887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "5fd6b4122b19433abab022a2bf3c697b"
format_stamp: "Formatted at 2026-08-08 07:01:54 on dist-test-slave-8tdl"
server_key: "6727eeb802da1000e053358da79b88d7"
server_key_iv: "455cad3fadf9bc2ba6db01601ffc0e83"
server_key_version: "encryptionkey@0"
I20260808 07:01:54.022281 17887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "5fd6b4122b19433abab022a2bf3c697b"
format_stamp: "Formatted at 2026-08-08 07:01:54 on dist-test-slave-8tdl"
server_key: "6727eeb802da1000e053358da79b88d7"
server_key_iv: "455cad3fadf9bc2ba6db01601ffc0e83"
server_key_version: "encryptionkey@0"
I20260808 07:01:54.023913 17887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:01:54.025034 17903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:54.025326 17887 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:54.025512 17887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "5fd6b4122b19433abab022a2bf3c697b"
format_stamp: "Formatted at 2026-08-08 07:01:54 on dist-test-slave-8tdl"
server_key: "6727eeb802da1000e053358da79b88d7"
server_key_iv: "455cad3fadf9bc2ba6db01601ffc0e83"
server_key_version: "encryptionkey@0"
I20260808 07:01:54.025604 17887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:54.049368 17887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:54.052487 17887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:54.052733 17887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:54.053050 17887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:54.053427 17887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:54.053475 17887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:54.053546 17887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:54.053812 17887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:54.061343 17887 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:45615
I20260808 07:01:54.061393 18022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:45615 every 8 connection(s)
I20260808 07:01:54.061774 17887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:01:54.066150 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 17887
I20260808 07:01:54.066274 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:01:54.066495 14140 external_mini_cluster.cc:1428] Setting key 4d0dc49228f03a2aca791fa78db1a2fd
Aug 08 07:01:54 dist-test-slave-8tdl krb5kdc[17776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172514, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:54.074784 17863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:54.062706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:38869 (local address 127.13.207.62:37033)
0808 07:01:54.062765 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:54.062767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:54.063016 (+ 249us) server_negotiation.cc:407] Connection header received
0808 07:01:54.064031 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:54.064032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:54.064045 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:54.064121 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:54.065088 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:54.065642 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:54.067276 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:54.067603 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:54.067686 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:54.070906 (+ 3220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:54.070908 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:54.070909 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:54.070918 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:54.072958 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:54.073496 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:54.073498 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:54.073498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:54.073572 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:54.073904 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:54.073906 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:54.073906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:54.074097 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:54.074174 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:54.074502 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 07:01:54.074622 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:01:54.074890 18026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:54.062629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:37033 (local address 127.13.207.1:38869)
0808 07:01:54.062828 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:54.062873 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:54.063903 (+ 1030us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:54.064241 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:54.064245 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:54.064589 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:54.064933 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:54.064939 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:54.065745 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:54.065746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:54.067174 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:54.067177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:54.067719 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:54.067720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:54.067876 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:54.068530 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:54.068541 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:54.070791 (+ 2250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:54.073080 (+ 2289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:54.073091 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:54.073093 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:54.073384 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:54.073667 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:54.073668 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:54.073668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:54.073814 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:54.074174 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:54.074181 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:54.074323 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 07:01:54.074433 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 07:01:54.074581 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":119,"threads_started":1}
I20260808 07:01:54.076567 18024 heartbeater.cc:347] Connected to a master server at 127.13.207.62:37033
I20260808 07:01:54.076718 18024 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:54.076963 18024 heartbeater.cc:511] Master 127.13.207.62:37033 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.207.2@KRBTEST.COM; defaulting to no policy
I20260808 07:01:54.077610 17822 ts_manager.cc:194] Registered new tserver with Master: 5fd6b4122b19433abab022a2bf3c697b (127.13.207.1:45615)
I20260808 07:01:54.078284 17822 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:38869
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
I20260808 07:01:54.104130 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:37033
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:54.205955 18029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:54.206195 18029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:54.206226 18029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:54.207854 18029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:54.207931 18029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:54.207981 18029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:01:54.210054 18029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:37033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:54.210318 18029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:54.210744 18029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:54.216419 18037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:54.216419 18034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:54.216419 18035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:54.216645 18029 server_base.cc:1034] running on GCE node
I20260808 07:01:54.216943 18029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:54.217252 18029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:54.218480 18029 hybrid_clock.cc:648] HybridClock initialized: now 1786172514218424 us; error 76 us; skew 500 ppm
Aug 08 07:01:54 dist-test-slave-8tdl krb5kdc[17776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172514, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:54.221612 18029 init.cc:377] Logged in from keytab as kudu/127.13.207.2@KRBTEST.COM (short username kudu)
I20260808 07:01:54.222302 18029 webserver.cc:460] Webserver started at http://127.13.207.2:34685/ using document root <none> and password file <none>
I20260808 07:01:54.222594 18029 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:54.222676 18029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:54.222899 18029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:54.224023 18029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "1ccc1d3a86ee4bd090c3c742366184f5"
format_stamp: "Formatted at 2026-08-08 07:01:54 on dist-test-slave-8tdl"
server_key: "37e2d87fbe797eae9702416a92744eeb"
server_key_iv: "f501d67f4d619ac55daca95d6f10e146"
server_key_version: "encryptionkey@0"
I20260808 07:01:54.224427 18029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "1ccc1d3a86ee4bd090c3c742366184f5"
format_stamp: "Formatted at 2026-08-08 07:01:54 on dist-test-slave-8tdl"
server_key: "37e2d87fbe797eae9702416a92744eeb"
server_key_iv: "f501d67f4d619ac55daca95d6f10e146"
server_key_version: "encryptionkey@0"
I20260808 07:01:54.226224 18029 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 07:01:54.227608 18044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:54.227974 18029 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:01:54.228150 18029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "1ccc1d3a86ee4bd090c3c742366184f5"
format_stamp: "Formatted at 2026-08-08 07:01:54 on dist-test-slave-8tdl"
server_key: "37e2d87fbe797eae9702416a92744eeb"
server_key_iv: "f501d67f4d619ac55daca95d6f10e146"
server_key_version: "encryptionkey@0"
I20260808 07:01:54.228272 18029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:54.239243 18029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:54.241945 18029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:54.242144 18029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:54.242431 18029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:54.242846 18029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:54.242905 18029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:54.242944 18029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:54.242970 18029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:54.250820 18029 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:40149
I20260808 07:01:54.250837 18163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:40149 every 8 connection(s)
I20260808 07:01:54.251632 18029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:01:54 dist-test-slave-8tdl krb5kdc[17776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172514, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:54.261792 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 18029
I20260808 07:01:54.261929 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:01:54.262133 14140 external_mini_cluster.cc:1428] Setting key 1dc8f25594535484bd286b40b85e64c1
I20260808 07:01:54.264093 17863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:54.252847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:52533 (local address 127.13.207.62:37033)
0808 07:01:54.252949 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:54.252952 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:54.253185 (+ 233us) server_negotiation.cc:407] Connection header received
0808 07:01:54.254269 (+ 1084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:54.254271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:54.254283 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:54.254354 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:54.255197 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:54.255936 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:54.257077 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:54.257279 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:54.257350 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:54.259882 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:54.259884 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:54.259885 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:54.259893 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:54.261703 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:54.262524 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:54.262526 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:54.262526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:54.262607 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:54.263134 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:54.263135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:54.263135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:54.263321 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:54.263464 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:54.263918 (+ 454us) server_negotiation.cc:299] Negotiation successful
0808 07:01:54.264021 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 07:01:54.264427 18166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:54.252797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:37033 (local address 127.13.207.2:52533)
0808 07:01:54.253014 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:54.253066 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:54.254132 (+ 1066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:54.254426 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:54.254429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:54.254849 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:54.255075 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:54.255082 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:54.256084 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:54.256085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:54.256937 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:54.256939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:54.257373 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:54.257374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:54.257505 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:54.258151 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:54.258160 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:54.259760 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:54.261823 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:54.261835 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:54.261838 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:54.262314 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:54.262735 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:54.262737 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:54.262737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:54.262916 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:54.263480 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:54.263489 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:54.263694 (+ 205us) client_negotiation.cc:769] Sending connection context
0808 07:01:54.263829 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 07:01:54.264097 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":135,"threads_started":1}
I20260808 07:01:54.265144 18164 heartbeater.cc:347] Connected to a master server at 127.13.207.62:37033
I20260808 07:01:54.265275 18164 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:54.265501 18164 heartbeater.cc:511] Master 127.13.207.62:37033 requested a full tablet report, sending...
I20260808 07:01:54.265925 17822 ts_manager.cc:194] Registered new tserver with Master: 1ccc1d3a86ee4bd090c3c742366184f5 (127.13.207.2:40149)
I20260808 07:01:54.266332 17822 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:52533
WARNING: no policy specified for kudu/127.13.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
I20260808 07:01:54.296561 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:37033
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:54.389822 18169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:54.390105 18169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:54.390136 18169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:54.391522 18169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:54.391582 18169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:54.391611 18169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:01:54.393293 18169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:37033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:54.393517 18169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:54.393709 18169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:54.399497 18177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:54.399508 18174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:54.399523 18175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:54.399888 18169 server_base.cc:1034] running on GCE node
I20260808 07:01:54.400208 18169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:54.400496 18169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:54.401666 18169 hybrid_clock.cc:648] HybridClock initialized: now 1786172514401639 us; error 39 us; skew 500 ppm
Aug 08 07:01:54 dist-test-slave-8tdl krb5kdc[17776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172514, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:54.405020 18169 init.cc:377] Logged in from keytab as kudu/127.13.207.3@KRBTEST.COM (short username kudu)
I20260808 07:01:54.405668 18169 webserver.cc:460] Webserver started at http://127.13.207.3:43851/ using document root <none> and password file <none>
I20260808 07:01:54.405975 18169 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:54.406060 18169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:54.406239 18169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:54.407573 18169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "0e869e3d090d49ed98bbee528cf32b73"
format_stamp: "Formatted at 2026-08-08 07:01:54 on dist-test-slave-8tdl"
server_key: "be9ea66b9197e73456625cf036a0228f"
server_key_iv: "f746d64bcda7ecb1671d1a83866e651a"
server_key_version: "encryptionkey@0"
I20260808 07:01:54.408012 18169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "0e869e3d090d49ed98bbee528cf32b73"
format_stamp: "Formatted at 2026-08-08 07:01:54 on dist-test-slave-8tdl"
server_key: "be9ea66b9197e73456625cf036a0228f"
server_key_iv: "f746d64bcda7ecb1671d1a83866e651a"
server_key_version: "encryptionkey@0"
I20260808 07:01:54.409822 18169 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 07:01:54.411051 18184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:54.411672 18169 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:54.411914 18169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "0e869e3d090d49ed98bbee528cf32b73"
format_stamp: "Formatted at 2026-08-08 07:01:54 on dist-test-slave-8tdl"
server_key: "be9ea66b9197e73456625cf036a0228f"
server_key_iv: "f746d64bcda7ecb1671d1a83866e651a"
server_key_version: "encryptionkey@0"
I20260808 07:01:54.412042 18169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:54.432444 18169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:54.435953 18169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:54.436160 18169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:54.436465 18169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:54.436889 18169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:54.436952 18169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:54.436990 18169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:54.437036 18169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:54.443624 18169 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:42529
I20260808 07:01:54.443785 18302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:42529 every 8 connection(s)
I20260808 07:01:54.444108 18169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:01:54 dist-test-slave-8tdl krb5kdc[17776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172514, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:54.454020 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 18169
I20260808 07:01:54.454108 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:01:54.454321 14140 external_mini_cluster.cc:1428] Setting key 94b48c41bbbdcd1e7c4876da1c8a08a5
I20260808 07:01:54.456629 17863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:54.445208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:43257 (local address 127.13.207.62:37033)
0808 07:01:54.445334 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:54.445337 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:54.445575 (+ 238us) server_negotiation.cc:407] Connection header received
0808 07:01:54.446855 (+ 1280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:54.446856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:54.446873 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:54.446941 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:54.447711 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:54.448419 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:54.449533 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:54.449745 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:54.449840 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:54.452934 (+ 3094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:54.452936 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:54.452936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:54.452944 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:54.454873 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:54.455436 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:54.455437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:54.455437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:54.455538 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:54.455855 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:54.455857 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:54.455857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:54.455992 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:54.456097 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:54.456414 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 07:01:54.456531 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 07:01:54.456884 18305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:54.445112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:37033 (local address 127.13.207.3:43257)
0808 07:01:54.445382 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:54.445431 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:54.446645 (+ 1214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:54.447044 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:54.447048 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:54.447363 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:54.447598 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:54.447602 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:54.448567 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:54.448568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:54.449427 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:54.449429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:54.449855 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:54.449856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:54.449997 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:54.450637 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:54.450647 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:54.452704 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:54.454997 (+ 2293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:54.455006 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:54.455009 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:54.455339 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:54.455617 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:54.455618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:54.455618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:54.455735 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:54.456101 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:54.456110 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:54.456269 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 07:01:54.456393 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 07:01:54.456582 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":97,"threads_started":1}
I20260808 07:01:54.457438 18303 heartbeater.cc:347] Connected to a master server at 127.13.207.62:37033
I20260808 07:01:54.457559 18303 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:54.457751 18303 heartbeater.cc:511] Master 127.13.207.62:37033 requested a full tablet report, sending...
I20260808 07:01:54.458189 17822 ts_manager.cc:194] Registered new tserver with Master: 0e869e3d090d49ed98bbee528cf32b73 (127.13.207.3:42529)
I20260808 07:01:54.458985 17822 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:43257
I20260808 07:01:54.466953 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 07:01:54 dist-test-slave-8tdl krb5kdc[17776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:01:54 dist-test-slave-8tdl krb5kdc[17776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.1@KRBTEST.COM
I20260808 07:01:54.497807 18310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:54.487259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44438 (local address 127.13.207.1:45615)
0808 07:01:54.487502 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:54.487512 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:54.487524 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:01:54.487552 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:54.487554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:54.487569 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:54.487651 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:54.488505 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:54.489313 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:54.490195 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:54.490593 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:54.490659 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:54.493737 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:54.493751 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:54.493751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:54.493760 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:54.495877 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:54.496625 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:54.496629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:54.496630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:54.496708 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:54.497035 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:54.497037 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:54.497037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:54.497225 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:54.497360 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:54.497594 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 07:01:54.497686 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":164,"threads_started":1}
Aug 08 07:01:54 dist-test-slave-8tdl krb5kdc[17776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:54.509392 17863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:54.499905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49564 (local address 127.13.207.62:37033)
0808 07:01:54.499988 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:54.499991 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:54.500000 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:01:54.500029 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:54.500030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:54.500042 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:54.500104 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:54.500714 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:54.501283 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:54.502115 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:54.502375 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:54.502468 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:54.505769 (+ 3301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:54.505771 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:54.505771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:54.505780 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:54.507684 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:54.508220 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:54.508222 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:54.508222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:54.508295 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:54.508633 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:54.508635 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:54.508635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:54.508931 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:54.509036 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:54.509219 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 07:01:54.509306 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 07:01:54.509761 17822 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49564
I20260808 07:01:54.510466 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 17887
I20260808 07:01:54.517271 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 18029
I20260808 07:01:54.523860 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 18169
I20260808 07:01:54.530812 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 17789
2026-08-08T07:01:54Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2127 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:01:54 dist-test-slave-8tdl krb5kdc[18315](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:01:54 dist-test-slave-8tdl krb5kdc[18315](info): set up 2 sockets
Aug 08 07:01:54 dist-test-slave-8tdl krb5kdc[18315](info): commencing operation
krb5kdc: starting...
W20260808 07:01:55.735069 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:01:55 dist-test-slave-8tdl krb5kdc[18315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:01:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:01:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:01:55.895051 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:37535
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:36101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:37535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:55.992612 18328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:55.993106 18328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:55.993204 18328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:55.994632 18328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:01:55.994745 18328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:55.994774 18328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:55.994796 18328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:01:55.994861 18328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:01:55.996693 18328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:36101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:37535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:37535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:55.997017 18328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:55.997318 18328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:01:56.003947 18328 server_base.cc:1034] running on GCE node
W20260808 07:01:56.003923 18334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:56.003923 18336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:56.003923 18333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:56.004510 18328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:56.004885 18328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:56.006064 18328 hybrid_clock.cc:648] HybridClock initialized: now 1786172516006050 us; error 41 us; skew 500 ppm
Aug 08 07:01:56 dist-test-slave-8tdl krb5kdc[18315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172516, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:56.009518 18328 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:01:56.010003 18328 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:56.010061 18328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:56.010207 18328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:56.011487 18328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "350a2e8fd3d4447c99df1c85276231e5"
format_stamp: "Formatted at 2026-08-08 07:01:56 on dist-test-slave-8tdl"
server_key: "e3219743e940ca4e20d4a1d20e0340a3"
server_key_iv: "571bd5774f87a8dad2b5253af87462e3"
server_key_version: "encryptionkey@0"
I20260808 07:01:56.011878 18328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "350a2e8fd3d4447c99df1c85276231e5"
format_stamp: "Formatted at 2026-08-08 07:01:56 on dist-test-slave-8tdl"
server_key: "e3219743e940ca4e20d4a1d20e0340a3"
server_key_iv: "571bd5774f87a8dad2b5253af87462e3"
server_key_version: "encryptionkey@0"
I20260808 07:01:56.014376 18328 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 07:01:56.015614 18342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:56.015931 18328 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:01:56.016137 18328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "350a2e8fd3d4447c99df1c85276231e5"
format_stamp: "Formatted at 2026-08-08 07:01:56 on dist-test-slave-8tdl"
server_key: "e3219743e940ca4e20d4a1d20e0340a3"
server_key_iv: "571bd5774f87a8dad2b5253af87462e3"
server_key_version: "encryptionkey@0"
I20260808 07:01:56.016242 18328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:56.046474 18328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:56.049301 18328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:56.049590 18328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:56.055960 18328 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:37535
I20260808 07:01:56.056133 18396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:37535 every 8 connection(s)
I20260808 07:01:56.056399 18328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:01:56.057494 18397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:56.060245 18397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5: Bootstrap starting.
I20260808 07:01:56.060881 18397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:56.061147 18397 log.cc:824] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:56.061976 18397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5: No bootstrap required, opened a new log
I20260808 07:01:56.063012 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 18328
I20260808 07:01:56.063143 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:01:56.063333 14140 external_mini_cluster.cc:1428] Setting key c90bbd69c36ae0640afe8bf824296a89
I20260808 07:01:56.064170 18397 raft_consensus.cc:348] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350a2e8fd3d4447c99df1c85276231e5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 37535 } }
I20260808 07:01:56.064342 18397 raft_consensus.cc:374] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:56.064368 18397 raft_consensus.cc:729] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 350a2e8fd3d4447c99df1c85276231e5, State: Initialized, Role: FOLLOWER
I20260808 07:01:56.064507 18397 consensus_queue.cc:260] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350a2e8fd3d4447c99df1c85276231e5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 37535 } }
I20260808 07:01:56.064630 18397 raft_consensus.cc:388] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:01:56.064679 18397 raft_consensus.cc:482] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:01:56.064738 18397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:56.065657 18397 raft_consensus.cc:504] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350a2e8fd3d4447c99df1c85276231e5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 37535 } }
I20260808 07:01:56.065801 18397 leader_election.cc:302] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 350a2e8fd3d4447c99df1c85276231e5; no voters:
I20260808 07:01:56.066020 18397 leader_election.cc:288] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:01:56.066085 18402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:56.066227 18402 raft_consensus.cc:686] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5 [term 1 LEADER]: Becoming Leader. State: Replica: 350a2e8fd3d4447c99df1c85276231e5, State: Running, Role: LEADER
I20260808 07:01:56.066336 18397 sys_catalog.cc:564] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:01:56.066323 18402 consensus_queue.cc:237] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350a2e8fd3d4447c99df1c85276231e5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 37535 } }
I20260808 07:01:56.066897 18404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 350a2e8fd3d4447c99df1c85276231e5. Latest consensus state: current_term: 1 leader_uuid: "350a2e8fd3d4447c99df1c85276231e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350a2e8fd3d4447c99df1c85276231e5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 37535 } } }
I20260808 07:01:56.067000 18404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:56.067154 18403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "350a2e8fd3d4447c99df1c85276231e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350a2e8fd3d4447c99df1c85276231e5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 37535 } } }
I20260808 07:01:56.067202 18403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:56.067291 18410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:01:56.067830 18410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:01:56.069943 18410 catalog_manager.cc:1347] Generated new cluster ID: edcb250a6f5c49979a57dd4be928acdf
I20260808 07:01:56.070080 18410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:01:56 dist-test-slave-8tdl krb5kdc[18315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:56.076403 18401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:56.063997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42212 (local address 127.13.207.62:37535)
0808 07:01:56.064562 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:56.064578 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:56.064606 (+ 28us) server_negotiation.cc:407] Connection header received
0808 07:01:56.064758 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:56.064767 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:56.064852 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:56.064990 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:56.065886 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.067463 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.068578 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.069188 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.069317 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:56.072240 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:56.072244 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:56.072245 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:56.072259 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:56.074108 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:56.074547 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:56.074551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:56.074551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:56.075164 (+ 613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:56.075417 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:56.075418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:56.075419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:56.075709 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:56.075904 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:56.076009 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 07:01:56.076172 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":477,"threads_started":1}
I20260808 07:01:56.085994 18410 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:01:56.086551 18410 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:01:56.092655 18410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5: Generated new TSK 0
I20260808 07:01:56.092913 18410 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
I20260808 07:01:56.148113 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:37535
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:36101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:56.248440 18424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:56.248692 18424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:56.248721 18424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:56.250051 18424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:56.250118 18424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:56.250150 18424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:01:56.251976 18424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:36101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:37535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:56.252300 18424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:56.252604 18424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:56.258621 18432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:56.258782 18424 server_base.cc:1034] running on GCE node
W20260808 07:01:56.258631 18429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:56.258644 18430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:56.259270 18424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:56.259681 18424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:56.260907 18424 hybrid_clock.cc:648] HybridClock initialized: now 1786172516260902 us; error 43 us; skew 500 ppm
Aug 08 07:01:56 dist-test-slave-8tdl krb5kdc[18315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172516, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:56.263796 18424 init.cc:377] Logged in from keytab as kudu/127.13.207.1@KRBTEST.COM (short username kudu)
I20260808 07:01:56.264240 18424 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:56.264292 18424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:56.264389 18424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:56.265533 18424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "3e4f8ca6417a4666a73b160cf71b2c93"
format_stamp: "Formatted at 2026-08-08 07:01:56 on dist-test-slave-8tdl"
server_key: "276014b9affdd1cbd90250e731f3dee2"
server_key_iv: "2eea75db15c1fa33bb545c0d36af0e47"
server_key_version: "encryptionkey@0"
I20260808 07:01:56.265849 18424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "3e4f8ca6417a4666a73b160cf71b2c93"
format_stamp: "Formatted at 2026-08-08 07:01:56 on dist-test-slave-8tdl"
server_key: "276014b9affdd1cbd90250e731f3dee2"
server_key_iv: "2eea75db15c1fa33bb545c0d36af0e47"
server_key_version: "encryptionkey@0"
I20260808 07:01:56.267894 18424 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 07:01:56.268783 18438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:56.269024 18424 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:56.269131 18424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "3e4f8ca6417a4666a73b160cf71b2c93"
format_stamp: "Formatted at 2026-08-08 07:01:56 on dist-test-slave-8tdl"
server_key: "276014b9affdd1cbd90250e731f3dee2"
server_key_iv: "2eea75db15c1fa33bb545c0d36af0e47"
server_key_version: "encryptionkey@0"
I20260808 07:01:56.269200 18424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:56.284114 18424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:56.286610 18424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:56.286870 18424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:56.287195 18424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:56.287659 18424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:56.287715 18424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:56.287741 18424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:56.287812 18424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:56.294922 18424 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:43857
I20260808 07:01:56.294993 18558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:43857 every 8 connection(s)
I20260808 07:01:56.295459 18424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:01:56.296191 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 18424
I20260808 07:01:56.296320 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:01:56.296551 14140 external_mini_cluster.cc:1428] Setting key 0d4a3e9385d7fbe1f3287acd1bd9f4c8
Aug 08 07:01:56 dist-test-slave-8tdl krb5kdc[18315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172516, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.207.2@KRBTEST.COM; defaulting to no policy
I20260808 07:01:56.307433 18401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:56.296374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:59537 (local address 127.13.207.62:37535)
0808 07:01:56.296756 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:56.296758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:56.296782 (+ 24us) server_negotiation.cc:407] Connection header received
0808 07:01:56.297483 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:56.297484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:56.297495 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:56.297559 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:56.299245 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.299754 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.300913 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.301147 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.301233 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:56.303957 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:56.303959 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:56.303959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:56.303971 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:56.305762 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:56.306258 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:56.306260 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:56.306260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:56.306329 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:56.306750 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:56.306751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:56.306751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:56.306889 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:56.306950 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:56.307268 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 07:01:56.307351 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354}
I20260808 07:01:56.307732 18562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:56.296393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:37535 (local address 127.13.207.1:59537)
0808 07:01:56.296619 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:56.296676 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:56.297369 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:56.297642 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:56.297647 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:56.298119 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:56.299097 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:56.299105 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.299882 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.299883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:56.300800 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:56.300803 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.301247 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.301248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:56.301384 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:56.302012 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:56.302021 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:56.303834 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:56.305895 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:56.305905 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:56.305907 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:56.306167 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:56.306441 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:56.306442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:56.306443 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:56.306621 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:56.306956 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:56.306963 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:56.307099 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 07:01:56.307189 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 07:01:56.307329 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":112,"threads_started":1}
I20260808 07:01:56.308357 18560 heartbeater.cc:347] Connected to a master server at 127.13.207.62:37535
I20260808 07:01:56.308451 18560 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:56.308655 18560 heartbeater.cc:511] Master 127.13.207.62:37535 requested a full tablet report, sending...
I20260808 07:01:56.309581 18360 ts_manager.cc:194] Registered new tserver with Master: 3e4f8ca6417a4666a73b160cf71b2c93 (127.13.207.1:43857)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.2@KRBTEST.COM" created.
I20260808 07:01:56.310308 18360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:59537
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
I20260808 07:01:56.331079 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:37535
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:36101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:56.427201 18565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:56.427457 18565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:56.427485 18565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:56.428692 18565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:56.428738 18565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:56.428768 18565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:01:56.430506 18565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:36101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:37535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:56.430792 18565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:56.431011 18565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:56.436311 18570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:56.436393 18573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:56.436509 18571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:56.436805 18565 server_base.cc:1034] running on GCE node
I20260808 07:01:56.437101 18565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:56.437407 18565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:56.438613 18565 hybrid_clock.cc:648] HybridClock initialized: now 1786172516438576 us; error 53 us; skew 500 ppm
Aug 08 07:01:56 dist-test-slave-8tdl krb5kdc[18315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172516, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:56.441623 18565 init.cc:377] Logged in from keytab as kudu/127.13.207.2@KRBTEST.COM (short username kudu)
I20260808 07:01:56.442058 18565 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:56.442148 18565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:56.442327 18565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:56.443742 18565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "a71e744c9e144347ba2d97f8b64162ea"
format_stamp: "Formatted at 2026-08-08 07:01:56 on dist-test-slave-8tdl"
server_key: "3dc186512d40583be3b8650d793c1559"
server_key_iv: "1ad19db9966d4ba726af40659ddadf56"
server_key_version: "encryptionkey@0"
I20260808 07:01:56.444176 18565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "a71e744c9e144347ba2d97f8b64162ea"
format_stamp: "Formatted at 2026-08-08 07:01:56 on dist-test-slave-8tdl"
server_key: "3dc186512d40583be3b8650d793c1559"
server_key_iv: "1ad19db9966d4ba726af40659ddadf56"
server_key_version: "encryptionkey@0"
I20260808 07:01:56.445808 18565 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:01:56.446907 18579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:56.447150 18565 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:56.447269 18565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "a71e744c9e144347ba2d97f8b64162ea"
format_stamp: "Formatted at 2026-08-08 07:01:56 on dist-test-slave-8tdl"
server_key: "3dc186512d40583be3b8650d793c1559"
server_key_iv: "1ad19db9966d4ba726af40659ddadf56"
server_key_version: "encryptionkey@0"
I20260808 07:01:56.447371 18565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:56.456925 18565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:56.459611 18565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:56.459836 18565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:56.460136 18565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:56.460527 18565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:56.460592 18565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:56.460644 18565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:56.460677 18565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:56.468081 18565 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:38741
I20260808 07:01:56.468120 18696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:38741 every 8 connection(s)
I20260808 07:01:56.468472 18565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:01:56.471230 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 18565
I20260808 07:01:56.471397 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:01:56.471639 14140 external_mini_cluster.cc:1428] Setting key 17ebac7b076a7211c9924f2753163f73
Aug 08 07:01:56 dist-test-slave-8tdl krb5kdc[18315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172516, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:56.482442 18401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:56.469327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:34469 (local address 127.13.207.62:37535)
0808 07:01:56.469446 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:56.469449 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:56.469653 (+ 204us) server_negotiation.cc:407] Connection header received
0808 07:01:56.470224 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:56.470226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:56.470238 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:56.470311 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:56.471925 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.472672 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.474697 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.475177 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.475286 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:56.478491 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:56.478493 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:56.478493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:56.478502 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:56.480255 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:56.480998 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:56.481000 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:56.481000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:56.481160 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:56.481553 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:56.481554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:56.481554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:56.481691 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:56.481755 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:56.482246 (+ 491us) server_negotiation.cc:299] Negotiation successful
0808 07:01:56.482341 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:01:56.482746 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:56.469251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:37535 (local address 127.13.207.2:34469)
0808 07:01:56.469462 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:56.469511 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:56.470100 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:56.470390 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:56.470394 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:56.470802 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:56.471757 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:56.471767 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.472839 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.472840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:56.474431 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:56.474436 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.475314 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.475315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:56.475469 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:56.476173 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:56.476185 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:56.478324 (+ 2139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:56.480373 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:56.480387 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:56.480392 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:56.480792 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:56.481262 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:56.481263 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:56.481264 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:56.481459 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:56.481774 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:56.481783 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:56.482016 (+ 233us) client_negotiation.cc:769] Sending connection context
0808 07:01:56.482173 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 07:01:56.482409 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":137,"threads_started":1}
WARNING: no policy specified for kudu/127.13.207.3@KRBTEST.COM; defaulting to no policy
I20260808 07:01:56.483361 18697 heartbeater.cc:347] Connected to a master server at 127.13.207.62:37535
I20260808 07:01:56.483520 18697 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:56.483716 18697 heartbeater.cc:511] Master 127.13.207.62:37535 requested a full tablet report, sending...
I20260808 07:01:56.484297 18360 ts_manager.cc:194] Registered new tserver with Master: a71e744c9e144347ba2d97f8b64162ea (127.13.207.2:38741)
I20260808 07:01:56.484731 18360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:34469
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
I20260808 07:01:56.507712 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:37535
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:36101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:01:56.598059 18703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:56.598358 18703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:56.598403 18703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:56.599823 18703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:56.599912 18703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:56.599968 18703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:01:56.601742 18703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:36101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:37535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:56.602066 18703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:56.602324 18703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:56.608021 18708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:56.608021 18711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:56.608075 18709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:56.608601 18703 server_base.cc:1034] running on GCE node
I20260808 07:01:56.608937 18703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:56.609234 18703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:56.610430 18703 hybrid_clock.cc:648] HybridClock initialized: now 1786172516610400 us; error 44 us; skew 500 ppm
Aug 08 07:01:56 dist-test-slave-8tdl krb5kdc[18315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172516, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:01:56.613557 18703 init.cc:377] Logged in from keytab as kudu/127.13.207.3@KRBTEST.COM (short username kudu)
I20260808 07:01:56.614041 18703 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:56.614126 18703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:56.614285 18703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:56.615499 18703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "5651adb248854920ab09d9468172e0b3"
format_stamp: "Formatted at 2026-08-08 07:01:56 on dist-test-slave-8tdl"
server_key: "9fdcac695239792bb823671a2eba9ba6"
server_key_iv: "fbd3c0de0539db3e6e3686a93005c223"
server_key_version: "encryptionkey@0"
I20260808 07:01:56.615869 18703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "5651adb248854920ab09d9468172e0b3"
format_stamp: "Formatted at 2026-08-08 07:01:56 on dist-test-slave-8tdl"
server_key: "9fdcac695239792bb823671a2eba9ba6"
server_key_iv: "fbd3c0de0539db3e6e3686a93005c223"
server_key_version: "encryptionkey@0"
I20260808 07:01:56.617273 18703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:01:56.618110 18717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:56.618332 18703 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:56.618445 18703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "5651adb248854920ab09d9468172e0b3"
format_stamp: "Formatted at 2026-08-08 07:01:56 on dist-test-slave-8tdl"
server_key: "9fdcac695239792bb823671a2eba9ba6"
server_key_iv: "fbd3c0de0539db3e6e3686a93005c223"
server_key_version: "encryptionkey@0"
I20260808 07:01:56.618525 18703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:56.635241 18703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:56.637979 18703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:56.638116 18703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:56.638335 18703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:56.638801 18703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:56.638847 18703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:56.638873 18703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:56.638892 18703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:56.645268 18703 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:40075
I20260808 07:01:56.645418 18835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:40075 every 8 connection(s)
I20260808 07:01:56.645649 18703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:01:56 dist-test-slave-8tdl krb5kdc[18315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172516, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:01:56.654433 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 18703
I20260808 07:01:56.654546 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:01:56.654829 14140 external_mini_cluster.cc:1428] Setting key b5f686437813530192094d300490b18c
I20260808 07:01:56.656621 18401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:56.646449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:43191 (local address 127.13.207.62:37535)
0808 07:01:56.646522 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:56.646524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:56.646779 (+ 255us) server_negotiation.cc:407] Connection header received
0808 07:01:56.647416 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:56.647417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:56.647430 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:56.647498 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:56.648554 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.649059 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.650221 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.650490 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.650616 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:56.653182 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:56.653183 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:56.653184 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:56.653191 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:56.655000 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:56.655519 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:56.655521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:56.655521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:56.655594 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:56.655968 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:56.655969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:56.655969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:56.656145 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:56.656205 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:56.656479 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 07:01:56.656553 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:01:56.656827 18838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:56.646438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:37535 (local address 127.13.207.3:43191)
0808 07:01:56.646601 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:56.646638 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:56.647254 (+ 616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:56.647575 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:56.647579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:56.647862 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:56.648429 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:56.648433 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.649194 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.649195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:56.650112 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:56.650115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.650634 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.650635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:56.650798 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:56.651460 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:01:56.651469 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:56.653076 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:56.655082 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:56.655093 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:56.655096 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:56.655416 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:56.655669 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:01:56.655670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:01:56.655670 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:01:56.655824 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:01:56.656198 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:56.656203 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:56.656332 (+ 129us) client_negotiation.cc:769] Sending connection context
0808 07:01:56.656424 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 07:01:56.656573 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
I20260808 07:01:56.657322 18836 heartbeater.cc:347] Connected to a master server at 127.13.207.62:37535
I20260808 07:01:56.657435 18836 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:56.657600 18836 heartbeater.cc:511] Master 127.13.207.62:37535 requested a full tablet report, sending...
I20260808 07:01:56.658036 18360 ts_manager.cc:194] Registered new tserver with Master: 5651adb248854920ab09d9468172e0b3 (127.13.207.3:40075)
I20260808 07:01:56.658417 18360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:43191
I20260808 07:01:56.667534 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:01:56.676941 18401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:56.669148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42220 (local address 127.13.207.62:37535)
0808 07:01:56.669264 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:56.669267 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:56.669369 (+ 102us) server_negotiation.cc:407] Connection header received
0808 07:01:56.669470 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:56.669471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:56.669487 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:56.669548 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:56.670189 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.670753 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.671544 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.671753 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.671842 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:56.673286 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:56.673288 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:56.673288 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:01:56.673295 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:56.675265 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:56.675704 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:56.675705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:56.675705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:56.675769 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:01:56.676257 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:01:56.676258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:01:56.676258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:01:56.676448 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:56.676569 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:56.676728 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 07:01:56.676829 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 07:01:56.679282 18360 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:01:56.681000 18360 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:01:56.689023 18852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:56.684394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47266 (local address 127.13.207.3:40075)
0808 07:01:56.684592 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:56.684629 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:56.684823 (+ 194us) server_negotiation.cc:407] Connection header received
0808 07:01:56.684876 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:56.684879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:56.684899 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:56.685016 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:56.685394 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.686239 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.687698 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.688471 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.688559 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:56.688631 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:56.688874 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 07:01:56.688921 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":91,"threads_started":1}
I20260808 07:01:56.689023 18848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:56.684009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:40075 (local address 127.0.0.1:47266)
0808 07:01:56.684569 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:56.684583 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:56.684657 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:56.685152 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:56.685156 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:56.685158 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:56.685276 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:56.685278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.686361 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.686362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:56.687591 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:56.687595 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.688579 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.688580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:56.688659 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:56.688671 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 07:01:56.688849 (+ 178us) client_negotiation.cc:240] Negotiation successful
0808 07:01:56.688917 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":33,"thread_start_us":164,"threads_started":1}
I20260808 07:01:56.690158 18847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:56.684007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:38741 (local address 127.0.0.1:48310)
0808 07:01:56.684246 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:56.684294 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:56.684374 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:56.685162 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:56.685164 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:56.685167 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:56.685250 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:56.685252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.686948 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.686949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:56.688252 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:56.688254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.689846 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.689847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:56.689944 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:56.689949 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:01:56.690007 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 07:01:56.690067 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":63,"spinlock_wait_cycles":768,"thread_start_us":203,"threads_started":1}
I20260808 07:01:56.690141 18767 tablet_service.cc:1467] Processing CreateTablet for tablet 93f25aea9fc4416e90ea2f630918ad7c (DEFAULT_TABLE table=test-table [id=6a3e0ae1f6664c9dbf23bb021a480caf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:56.690388 18851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:56.684338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48310 (local address 127.13.207.2:38741)
0808 07:01:56.684963 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:56.684967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:56.684978 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:01:56.685015 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:56.685017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:56.685033 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:56.685439 (+ 406us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:56.685540 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.686810 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.688377 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.689747 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.689845 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:56.689926 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:56.690117 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 07:01:56.690175 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":126,"threads_started":1}
I20260808 07:01:56.690573 18767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93f25aea9fc4416e90ea2f630918ad7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:56.690593 18850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:56.684247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:43857 (local address 127.0.0.1:48500)
0808 07:01:56.684699 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:56.684709 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:56.684773 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:56.685154 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:56.685157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:56.685159 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:56.685275 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:56.685278 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.686945 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.686946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:56.688221 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:56.688225 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.690004 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.690005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:56.690087 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:56.690093 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:01:56.690149 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 07:01:56.690197 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":6,"thread_start_us":95,"threads_started":1}
I20260808 07:01:56.691017 18853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:56.684518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48500 (local address 127.13.207.1:43857)
0808 07:01:56.684690 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:56.684695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:56.684758 (+ 63us) server_negotiation.cc:407] Connection header received
0808 07:01:56.684858 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:56.684862 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:56.684996 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:56.685084 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:56.685416 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.686818 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.688800 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.689914 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.689987 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:56.690049 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:56.690874 (+ 825us) server_negotiation.cc:299] Negotiation successful
0808 07:01:56.690929 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
I20260808 07:01:56.691159 18630 tablet_service.cc:1467] Processing CreateTablet for tablet 93f25aea9fc4416e90ea2f630918ad7c (DEFAULT_TABLE table=test-table [id=6a3e0ae1f6664c9dbf23bb021a480caf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:56.691547 18630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93f25aea9fc4416e90ea2f630918ad7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:56.691715 18490 tablet_service.cc:1467] Processing CreateTablet for tablet 93f25aea9fc4416e90ea2f630918ad7c (DEFAULT_TABLE table=test-table [id=6a3e0ae1f6664c9dbf23bb021a480caf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:56.692065 18490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93f25aea9fc4416e90ea2f630918ad7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:56.693800 18854 tablet_bootstrap.cc:492] T 93f25aea9fc4416e90ea2f630918ad7c P 5651adb248854920ab09d9468172e0b3: Bootstrap starting.
I20260808 07:01:56.694525 18854 tablet_bootstrap.cc:654] T 93f25aea9fc4416e90ea2f630918ad7c P 5651adb248854920ab09d9468172e0b3: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:56.695164 18854 log.cc:824] T 93f25aea9fc4416e90ea2f630918ad7c P 5651adb248854920ab09d9468172e0b3: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:56.695495 18855 tablet_bootstrap.cc:492] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea: Bootstrap starting.
I20260808 07:01:56.695906 18857 tablet_bootstrap.cc:492] T 93f25aea9fc4416e90ea2f630918ad7c P 3e4f8ca6417a4666a73b160cf71b2c93: Bootstrap starting.
I20260808 07:01:56.696339 18855 tablet_bootstrap.cc:654] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:56.696609 18854 tablet_bootstrap.cc:492] T 93f25aea9fc4416e90ea2f630918ad7c P 5651adb248854920ab09d9468172e0b3: No bootstrap required, opened a new log
I20260808 07:01:56.696615 18857 tablet_bootstrap.cc:654] T 93f25aea9fc4416e90ea2f630918ad7c P 3e4f8ca6417a4666a73b160cf71b2c93: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:56.696762 18854 ts_tablet_manager.cc:1397] T 93f25aea9fc4416e90ea2f630918ad7c P 5651adb248854920ab09d9468172e0b3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:01:56.697016 18855 log.cc:824] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:56.697290 18857 log.cc:824] T 93f25aea9fc4416e90ea2f630918ad7c P 3e4f8ca6417a4666a73b160cf71b2c93: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:56.697944 18855 tablet_bootstrap.cc:492] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea: No bootstrap required, opened a new log
I20260808 07:01:56.698031 18855 ts_tablet_manager.cc:1397] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:01:56.699424 18857 tablet_bootstrap.cc:492] T 93f25aea9fc4416e90ea2f630918ad7c P 3e4f8ca6417a4666a73b160cf71b2c93: No bootstrap required, opened a new log
I20260808 07:01:56.699522 18857 ts_tablet_manager.cc:1397] T 93f25aea9fc4416e90ea2f630918ad7c P 3e4f8ca6417a4666a73b160cf71b2c93: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 07:01:56.699301 18854 raft_consensus.cc:348] T 93f25aea9fc4416e90ea2f630918ad7c P 5651adb248854920ab09d9468172e0b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5651adb248854920ab09d9468172e0b3" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40075 } } peers { permanent_uuid: "a71e744c9e144347ba2d97f8b64162ea" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38741 } } peers { permanent_uuid: "3e4f8ca6417a4666a73b160cf71b2c93" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 43857 } }
I20260808 07:01:56.699676 18854 raft_consensus.cc:374] T 93f25aea9fc4416e90ea2f630918ad7c P 5651adb248854920ab09d9468172e0b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:56.699744 18854 raft_consensus.cc:729] T 93f25aea9fc4416e90ea2f630918ad7c P 5651adb248854920ab09d9468172e0b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5651adb248854920ab09d9468172e0b3, State: Initialized, Role: FOLLOWER
I20260808 07:01:56.699913 18854 consensus_queue.cc:260] T 93f25aea9fc4416e90ea2f630918ad7c P 5651adb248854920ab09d9468172e0b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5651adb248854920ab09d9468172e0b3" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40075 } } peers { permanent_uuid: "a71e744c9e144347ba2d97f8b64162ea" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38741 } } peers { permanent_uuid: "3e4f8ca6417a4666a73b160cf71b2c93" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 43857 } }
I20260808 07:01:56.700294 18854 ts_tablet_manager.cc:1428] T 93f25aea9fc4416e90ea2f630918ad7c P 5651adb248854920ab09d9468172e0b3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:01:56.700304 18836 heartbeater.cc:503] Master 127.13.207.62:37535 was elected leader, sending a full tablet report...
I20260808 07:01:56.700662 18855 raft_consensus.cc:348] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5651adb248854920ab09d9468172e0b3" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40075 } } peers { permanent_uuid: "a71e744c9e144347ba2d97f8b64162ea" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38741 } } peers { permanent_uuid: "3e4f8ca6417a4666a73b160cf71b2c93" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 43857 } }
I20260808 07:01:56.700876 18855 raft_consensus.cc:374] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:56.700941 18855 raft_consensus.cc:729] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a71e744c9e144347ba2d97f8b64162ea, State: Initialized, Role: FOLLOWER
I20260808 07:01:56.701067 18855 consensus_queue.cc:260] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5651adb248854920ab09d9468172e0b3" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40075 } } peers { permanent_uuid: "a71e744c9e144347ba2d97f8b64162ea" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38741 } } peers { permanent_uuid: "3e4f8ca6417a4666a73b160cf71b2c93" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 43857 } }
I20260808 07:01:56.701583 18855 ts_tablet_manager.cc:1428] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 07:01:56.701622 18697 heartbeater.cc:503] Master 127.13.207.62:37535 was elected leader, sending a full tablet report...
I20260808 07:01:56.701757 18857 raft_consensus.cc:348] T 93f25aea9fc4416e90ea2f630918ad7c P 3e4f8ca6417a4666a73b160cf71b2c93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5651adb248854920ab09d9468172e0b3" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40075 } } peers { permanent_uuid: "a71e744c9e144347ba2d97f8b64162ea" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38741 } } peers { permanent_uuid: "3e4f8ca6417a4666a73b160cf71b2c93" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 43857 } }
I20260808 07:01:56.701923 18857 raft_consensus.cc:374] T 93f25aea9fc4416e90ea2f630918ad7c P 3e4f8ca6417a4666a73b160cf71b2c93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:56.701967 18857 raft_consensus.cc:729] T 93f25aea9fc4416e90ea2f630918ad7c P 3e4f8ca6417a4666a73b160cf71b2c93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e4f8ca6417a4666a73b160cf71b2c93, State: Initialized, Role: FOLLOWER
I20260808 07:01:56.702081 18857 consensus_queue.cc:260] T 93f25aea9fc4416e90ea2f630918ad7c P 3e4f8ca6417a4666a73b160cf71b2c93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5651adb248854920ab09d9468172e0b3" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40075 } } peers { permanent_uuid: "a71e744c9e144347ba2d97f8b64162ea" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38741 } } peers { permanent_uuid: "3e4f8ca6417a4666a73b160cf71b2c93" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 43857 } }
I20260808 07:01:56.702400 18857 ts_tablet_manager.cc:1428] T 93f25aea9fc4416e90ea2f630918ad7c P 3e4f8ca6417a4666a73b160cf71b2c93: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:01:56.702435 18560 heartbeater.cc:503] Master 127.13.207.62:37535 was elected leader, sending a full tablet report...
W20260808 07:01:56.719269 18698 tablet.cc:2369] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:01:56.737304 18862 raft_consensus.cc:482] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:01:56.737453 18862 raft_consensus.cc:504] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5651adb248854920ab09d9468172e0b3" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40075 } } peers { permanent_uuid: "a71e744c9e144347ba2d97f8b64162ea" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38741 } } peers { permanent_uuid: "3e4f8ca6417a4666a73b160cf71b2c93" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 43857 } }
I20260808 07:01:56.737809 18862 leader_election.cc:288] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5651adb248854920ab09d9468172e0b3 (127.13.207.3:40075), 3e4f8ca6417a4666a73b160cf71b2c93 (127.13.207.1:43857)
I20260808 07:01:56.741389 18852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:56.738206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:58781 (local address 127.13.207.3:40075)
0808 07:01:56.738334 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:56.738336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:56.738343 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:01:56.738379 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:56.738380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:56.738388 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:56.738452 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:56.738700 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.739251 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.740322 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.740823 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.741050 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:56.741062 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:56.741255 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 07:01:56.741299 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260808 07:01:56.741398 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:56.738079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:40075 (local address 127.13.207.2:58781)
0808 07:01:56.738142 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:56.738156 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:56.738203 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:56.738484 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:56.738485 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:56.738488 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:56.738560 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:56.738562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.739443 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.739445 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:56.740214 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:56.740216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.741054 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.741055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:56.741166 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:56.741170 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:01:56.741249 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 07:01:56.741299 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 07:01:56.741813 18853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:56.738211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:56041 (local address 127.13.207.1:43857)
0808 07:01:56.738300 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:56.738302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:56.738670 (+ 368us) server_negotiation.cc:407] Connection header received
0808 07:01:56.738799 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:56.738800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:56.738811 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:56.738912 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:01:56.739123 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.739625 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.740687 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.741249 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.741377 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:56.741388 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:56.741654 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 07:01:56.741729 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 07:01:56.741881 18788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93f25aea9fc4416e90ea2f630918ad7c" candidate_uuid: "a71e744c9e144347ba2d97f8b64162ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5651adb248854920ab09d9468172e0b3" is_pre_election: true
I20260808 07:01:56.742060 18864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:56.738079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:43857 (local address 127.13.207.2:56041)
0808 07:01:56.738585 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:56.738595 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:56.738677 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:56.738918 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:56.738919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:56.738921 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:01:56.739004 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:56.739005 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.739719 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.739721 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:56.740567 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:56.740569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.741378 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.741379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:56.741530 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:56.741536 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:01:56.741924 (+ 388us) client_negotiation.cc:240] Negotiation successful
0808 07:01:56.741975 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"mutex_wait_us":19,"thread_start_us":132,"threads_started":1}
I20260808 07:01:56.742096 18788 raft_consensus.cc:2454] T 93f25aea9fc4416e90ea2f630918ad7c P 5651adb248854920ab09d9468172e0b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a71e744c9e144347ba2d97f8b64162ea in term 0.
I20260808 07:01:56.742571 18582 leader_election.cc:302] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5651adb248854920ab09d9468172e0b3, a71e744c9e144347ba2d97f8b64162ea; no voters:
I20260808 07:01:56.742676 18512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93f25aea9fc4416e90ea2f630918ad7c" candidate_uuid: "a71e744c9e144347ba2d97f8b64162ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e4f8ca6417a4666a73b160cf71b2c93" is_pre_election: true
I20260808 07:01:56.742751 18862 raft_consensus.cc:2781] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:01:56.742832 18862 raft_consensus.cc:482] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:01:56.742861 18862 raft_consensus.cc:3037] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:56.742911 18512 raft_consensus.cc:2454] T 93f25aea9fc4416e90ea2f630918ad7c P 3e4f8ca6417a4666a73b160cf71b2c93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a71e744c9e144347ba2d97f8b64162ea in term 0.
I20260808 07:01:56.743659 18862 raft_consensus.cc:504] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5651adb248854920ab09d9468172e0b3" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40075 } } peers { permanent_uuid: "a71e744c9e144347ba2d97f8b64162ea" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38741 } } peers { permanent_uuid: "3e4f8ca6417a4666a73b160cf71b2c93" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 43857 } }
I20260808 07:01:56.743825 18862 leader_election.cc:288] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea [CANDIDATE]: Term 1 election: Requested vote from peers 5651adb248854920ab09d9468172e0b3 (127.13.207.3:40075), 3e4f8ca6417a4666a73b160cf71b2c93 (127.13.207.1:43857)
I20260808 07:01:56.743984 18788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93f25aea9fc4416e90ea2f630918ad7c" candidate_uuid: "a71e744c9e144347ba2d97f8b64162ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5651adb248854920ab09d9468172e0b3"
I20260808 07:01:56.744056 18512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93f25aea9fc4416e90ea2f630918ad7c" candidate_uuid: "a71e744c9e144347ba2d97f8b64162ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e4f8ca6417a4666a73b160cf71b2c93"
I20260808 07:01:56.744093 18788 raft_consensus.cc:3037] T 93f25aea9fc4416e90ea2f630918ad7c P 5651adb248854920ab09d9468172e0b3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:56.744210 18512 raft_consensus.cc:3037] T 93f25aea9fc4416e90ea2f630918ad7c P 3e4f8ca6417a4666a73b160cf71b2c93 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:56.744942 18788 raft_consensus.cc:2454] T 93f25aea9fc4416e90ea2f630918ad7c P 5651adb248854920ab09d9468172e0b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a71e744c9e144347ba2d97f8b64162ea in term 1.
I20260808 07:01:56.745102 18512 raft_consensus.cc:2454] T 93f25aea9fc4416e90ea2f630918ad7c P 3e4f8ca6417a4666a73b160cf71b2c93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a71e744c9e144347ba2d97f8b64162ea in term 1.
I20260808 07:01:56.745147 18582 leader_election.cc:302] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5651adb248854920ab09d9468172e0b3, a71e744c9e144347ba2d97f8b64162ea; no voters:
I20260808 07:01:56.745252 18862 raft_consensus.cc:2781] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:56.745489 18862 raft_consensus.cc:686] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea [term 1 LEADER]: Becoming Leader. State: Replica: a71e744c9e144347ba2d97f8b64162ea, State: Running, Role: LEADER
I20260808 07:01:56.745616 18862 consensus_queue.cc:237] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5651adb248854920ab09d9468172e0b3" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40075 } } peers { permanent_uuid: "a71e744c9e144347ba2d97f8b64162ea" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38741 } } peers { permanent_uuid: "3e4f8ca6417a4666a73b160cf71b2c93" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 43857 } }
I20260808 07:01:56.746587 18360 catalog_manager.cc:5495] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea reported cstate change: term changed from 0 to 1, leader changed from <none> to a71e744c9e144347ba2d97f8b64162ea (127.13.207.2). New cstate: current_term: 1 leader_uuid: "a71e744c9e144347ba2d97f8b64162ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5651adb248854920ab09d9468172e0b3" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a71e744c9e144347ba2d97f8b64162ea" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e4f8ca6417a4666a73b160cf71b2c93" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 43857 } health_report { overall_health: UNKNOWN } } }
I20260808 07:01:56.767766 18851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:56.764646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48322 (local address 127.13.207.2:38741)
0808 07:01:56.764713 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:56.764715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:56.764725 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:01:56.764771 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:56.764771 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:56.764791 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:56.764859 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:01:56.765038 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.765552 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.766622 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:56.766913 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:56.767023 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:56.767197 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:01:56.767326 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:01:56.767401 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:56.767501 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 07:01:56.767522 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:01:56.770030 18788 raft_consensus.cc:1261] T 93f25aea9fc4416e90ea2f630918ad7c P 5651adb248854920ab09d9468172e0b3 [term 1 FOLLOWER]: Refusing update from remote peer a71e744c9e144347ba2d97f8b64162ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:01:56.770078 18512 raft_consensus.cc:1261] T 93f25aea9fc4416e90ea2f630918ad7c P 3e4f8ca6417a4666a73b160cf71b2c93 [term 1 FOLLOWER]: Refusing update from remote peer a71e744c9e144347ba2d97f8b64162ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:01:56.770533 18865 consensus_queue.cc:1035] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "5651adb248854920ab09d9468172e0b3" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:01:56.770660 18862 consensus_queue.cc:1035] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e4f8ca6417a4666a73b160cf71b2c93" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 43857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:01:56.774248 18868 mvcc.cc:205] Tried to move back new op lower bound from 7316162628686508032 to 7316162628590686208. Current Snapshot: MvccSnapshot[applied={T|T < 7316162628686508032}]
I20260808 07:01:56.774410 18870 mvcc.cc:205] Tried to move back new op lower bound from 7316162628686508032 to 7316162628590686208. Current Snapshot: MvccSnapshot[applied={T|T < 7316162628686508032}]
I20260808 07:01:56.777808 18358 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:01:56.778048 18358 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:01:56.779163 18358 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 350a2e8fd3d4447c99df1c85276231e5: Sending DeleteTablet for 3 replicas of tablet 93f25aea9fc4416e90ea2f630918ad7c
I20260808 07:01:56.779711 18490 tablet_service.cc:1514] Processing DeleteTablet for tablet 93f25aea9fc4416e90ea2f630918ad7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:56 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:48500
I20260808 07:01:56.779767 18630 tablet_service.cc:1514] Processing DeleteTablet for tablet 93f25aea9fc4416e90ea2f630918ad7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:56 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:48310
I20260808 07:01:56.779788 18767 tablet_service.cc:1514] Processing DeleteTablet for tablet 93f25aea9fc4416e90ea2f630918ad7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:56 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:47266
I20260808 07:01:56.780100 18881 tablet_replica.cc:330] T 93f25aea9fc4416e90ea2f630918ad7c P 3e4f8ca6417a4666a73b160cf71b2c93: stopping tablet replica
I20260808 07:01:56.780239 18881 raft_consensus.cc:2229] T 93f25aea9fc4416e90ea2f630918ad7c P 3e4f8ca6417a4666a73b160cf71b2c93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:01:56.780233 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 18424
I20260808 07:01:56.780362 18881 raft_consensus.cc:2258] T 93f25aea9fc4416e90ea2f630918ad7c P 3e4f8ca6417a4666a73b160cf71b2c93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:01:56.781157 18879 tablet_replica.cc:330] T 93f25aea9fc4416e90ea2f630918ad7c P 5651adb248854920ab09d9468172e0b3: stopping tablet replica
I20260808 07:01:56.781407 18879 raft_consensus.cc:2229] T 93f25aea9fc4416e90ea2f630918ad7c P 5651adb248854920ab09d9468172e0b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:01:56.781577 18879 raft_consensus.cc:2258] T 93f25aea9fc4416e90ea2f630918ad7c P 5651adb248854920ab09d9468172e0b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:01:56.782228 18879 ts_tablet_manager.cc:1905] T 93f25aea9fc4416e90ea2f630918ad7c P 5651adb248854920ab09d9468172e0b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:01:56.784479 18879 ts_tablet_manager.cc:1918] T 93f25aea9fc4416e90ea2f630918ad7c P 5651adb248854920ab09d9468172e0b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:01:56.784576 18879 log.cc:1196] T 93f25aea9fc4416e90ea2f630918ad7c P 5651adb248854920ab09d9468172e0b3: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/93f25aea9fc4416e90ea2f630918ad7c
I20260808 07:01:56.784894 18879 ts_tablet_manager.cc:1939] T 93f25aea9fc4416e90ea2f630918ad7c P 5651adb248854920ab09d9468172e0b3: Deleting consensus metadata
I20260808 07:01:56.785497 18345 catalog_manager.cc:4841] TS 5651adb248854920ab09d9468172e0b3 (127.13.207.3:40075): tablet 93f25aea9fc4416e90ea2f630918ad7c (table test-table [id=6a3e0ae1f6664c9dbf23bb021a480caf]) successfully deleted
W20260808 07:01:56.790072 18343 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.207.1:43857 (error 108)
I20260808 07:01:56.790387 18880 tablet_replica.cc:330] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea: stopping tablet replica
I20260808 07:01:56.790645 18880 raft_consensus.cc:2229] T 93f25aea9fc4416e90ea2f630918ad7c P a71e744c9e144347ba2d97f8b64162ea [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:01:56.790829 18850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:56.790650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:43857 (local address 127.0.0.1:48504)
0808 07:01:56.790739 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:56.790780 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.1:43857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 07:01:56.790899 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 18565
W20260808 07:01:56.790968 18343 catalog_manager.cc:4584] TS 3e4f8ca6417a4666a73b160cf71b2c93 (127.13.207.1:43857): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 93f25aea9fc4416e90ea2f630918ad7c: Network error: Client connection negotiation failed: client connection to 127.13.207.1:43857: connect: Connection refused (error 111)
I20260808 07:01:56.797668 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 18703
I20260808 07:01:56.797890 18850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:56.797664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:38741 (local address 127.0.0.1:48332)
0808 07:01:56.797771 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:56.797840 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.2:38741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 07:01:56.804096 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 18328
2026-08-08T07:01:56Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2272 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T07:01:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:01:56Z Disabled control of system clock
I20260808 07:01:56.836901 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:34353
--webserver_interface=127.13.207.62
--webserver_port=0
--builtin_ntp_servers=127.13.207.20:44601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:34353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 07:01:56.927711 18887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:56.927955 18887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:56.927979 18887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:56.929193 18887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:01:56.929239 18887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:56.929253 18887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:56.929266 18887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:01:56.929279 18887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:01:56.930658 18887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:34353
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:34353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:56.930932 18887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:56.931210 18887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:56.937908 18892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:56.937908 18895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:56.938331 18887 server_base.cc:1034] running on GCE node
W20260808 07:01:56.937908 18893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:56.938800 18887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:56.939291 18887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:56.940483 18887 hybrid_clock.cc:648] HybridClock initialized: now 1786172516940461 us; error 61 us; skew 500 ppm
I20260808 07:01:56.942219 18887 webserver.cc:460] Webserver started at http://127.13.207.62:39625/ using document root <none> and password file <none>
I20260808 07:01:56.942545 18887 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:56.942605 18887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:56.942783 18887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:56.943990 18887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "d008f8157bf94b05a8a8a8c1d755fd0c"
format_stamp: "Formatted at 2026-08-08 07:01:56 on dist-test-slave-8tdl"
server_key: "55ad6ffcc61f481df91abb8813bf6a38"
server_key_iv: "000ef17e7c26a9b415d9102d4cb47f4f"
server_key_version: "encryptionkey@0"
I20260808 07:01:56.944398 18887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "d008f8157bf94b05a8a8a8c1d755fd0c"
format_stamp: "Formatted at 2026-08-08 07:01:56 on dist-test-slave-8tdl"
server_key: "55ad6ffcc61f481df91abb8813bf6a38"
server_key_iv: "000ef17e7c26a9b415d9102d4cb47f4f"
server_key_version: "encryptionkey@0"
I20260808 07:01:56.945904 18887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:01:56.946993 18901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:56.947217 18887 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:56.947319 18887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "d008f8157bf94b05a8a8a8c1d755fd0c"
format_stamp: "Formatted at 2026-08-08 07:01:56 on dist-test-slave-8tdl"
server_key: "55ad6ffcc61f481df91abb8813bf6a38"
server_key_iv: "000ef17e7c26a9b415d9102d4cb47f4f"
server_key_version: "encryptionkey@0"
I20260808 07:01:56.947386 18887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:56.951630 18887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:56.952196 18887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:56.952359 18887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:56.958068 18887 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:34353
I20260808 07:01:56.958072 18956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:34353 every 8 connection(s)
I20260808 07:01:56.958964 18887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:01:56.960148 18957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:56.962740 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 18887
I20260808 07:01:56.962879 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:01:56.963068 14140 external_mini_cluster.cc:1428] Setting key 7f8745d6ec356237d33091a239954012
I20260808 07:01:56.963794 18957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c: Bootstrap starting.
I20260808 07:01:56.964598 18957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:56.964979 18957 log.cc:824] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:56.966027 18957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c: No bootstrap required, opened a new log
I20260808 07:01:56.966567 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:56.963739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47604 (local address 127.13.207.62:34353)
0808 07:01:56.964051 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:56.964060 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:56.964092 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:01:56.964753 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:56.964766 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:56.964833 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:56.965006 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:56.965790 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:56.965794 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:56.965795 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:56.965800 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:56.965854 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:56.965868 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:56.965966 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:56.966264 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 07:01:56.966334 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":113,"threads_started":1}
I20260808 07:01:56.967937 18957 raft_consensus.cc:348] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d008f8157bf94b05a8a8a8c1d755fd0c" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 34353 } }
I20260808 07:01:56.968086 18957 raft_consensus.cc:374] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:56.968111 18957 raft_consensus.cc:729] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d008f8157bf94b05a8a8a8c1d755fd0c, State: Initialized, Role: FOLLOWER
I20260808 07:01:56.968235 18957 consensus_queue.cc:260] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d008f8157bf94b05a8a8a8c1d755fd0c" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 34353 } }
I20260808 07:01:56.969015 18957 raft_consensus.cc:388] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:01:56.969067 18957 raft_consensus.cc:482] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:01:56.969117 18957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:56.969776 18957 raft_consensus.cc:504] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d008f8157bf94b05a8a8a8c1d755fd0c" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 34353 } }
I20260808 07:01:56.969893 18957 leader_election.cc:302] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d008f8157bf94b05a8a8a8c1d755fd0c; no voters:
I20260808 07:01:56.970063 18957 leader_election.cc:288] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:01:56.970202 18962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:56.970369 18962 raft_consensus.cc:686] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c [term 1 LEADER]: Becoming Leader. State: Replica: d008f8157bf94b05a8a8a8c1d755fd0c, State: Running, Role: LEADER
I20260808 07:01:56.970443 18957 sys_catalog.cc:564] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:01:56.970480 18962 consensus_queue.cc:237] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d008f8157bf94b05a8a8a8c1d755fd0c" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 34353 } }
I20260808 07:01:56.971000 18963 sys_catalog.cc:455] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d008f8157bf94b05a8a8a8c1d755fd0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d008f8157bf94b05a8a8a8c1d755fd0c" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 34353 } } }
I20260808 07:01:56.971175 18963 sys_catalog.cc:458] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:56.971370 18964 sys_catalog.cc:455] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d008f8157bf94b05a8a8a8c1d755fd0c. Latest consensus state: current_term: 1 leader_uuid: "d008f8157bf94b05a8a8a8c1d755fd0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d008f8157bf94b05a8a8a8c1d755fd0c" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 34353 } } }
I20260808 07:01:56.971457 18964 sys_catalog.cc:458] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c [sys.catalog]: This master's current role is: LEADER
W20260808 07:01:56.972664 18979 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:01:56.972731 18979 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:01:56.972821 18970 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:01:56.973109 18970 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:01:56.975333 18970 catalog_manager.cc:1347] Generated new cluster ID: 5c84caa169104a75bd45a93c2dd139eb
I20260808 07:01:56.975407 18970 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:01:56.989123 18970 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:01:56.989362 18970 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:01:56.999935 18970 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c: Generated new TSK 0
I20260808 07:01:57.000241 18970 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:01:57.006556 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:34353
--builtin_ntp_servers=127.13.207.20:44601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 07:01:57.105240 18982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:57.105504 18982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:57.105535 18982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:57.107074 18982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:57.107131 18982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:57.107167 18982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:01:57.108841 18982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:34353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:57.109177 18982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:57.109563 18982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:57.116416 18987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:57.116472 18988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:57.116472 18990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:57.116781 18982 server_base.cc:1034] running on GCE node
I20260808 07:01:57.117007 18982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:57.117311 18982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:57.118495 18982 hybrid_clock.cc:648] HybridClock initialized: now 1786172517118423 us; error 85 us; skew 500 ppm
I20260808 07:01:57.120254 18982 webserver.cc:460] Webserver started at http://127.13.207.1:40155/ using document root <none> and password file <none>
I20260808 07:01:57.120590 18982 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:57.120657 18982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:57.120800 18982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:57.122548 18982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "56d93d4522f146f28193509788b9e179"
format_stamp: "Formatted at 2026-08-08 07:01:57 on dist-test-slave-8tdl"
server_key: "86cccb87e6c2a8c02c76d148c99620dc"
server_key_iv: "a5150d936397118aa655ecf151468302"
server_key_version: "encryptionkey@0"
I20260808 07:01:57.123112 18982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "56d93d4522f146f28193509788b9e179"
format_stamp: "Formatted at 2026-08-08 07:01:57 on dist-test-slave-8tdl"
server_key: "86cccb87e6c2a8c02c76d148c99620dc"
server_key_iv: "a5150d936397118aa655ecf151468302"
server_key_version: "encryptionkey@0"
I20260808 07:01:57.125068 18982 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 07:01:57.126837 18996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:57.127159 18982 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:57.127314 18982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "56d93d4522f146f28193509788b9e179"
format_stamp: "Formatted at 2026-08-08 07:01:57 on dist-test-slave-8tdl"
server_key: "86cccb87e6c2a8c02c76d148c99620dc"
server_key_iv: "a5150d936397118aa655ecf151468302"
server_key_version: "encryptionkey@0"
I20260808 07:01:57.127472 18982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:57.131917 18982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:57.132247 18982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:57.132418 18982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:57.132705 18982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:57.133093 18982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:57.133137 18982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:57.133162 18982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:57.133174 18982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:57.141026 18982 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:45963
I20260808 07:01:57.141183 19116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:45963 every 8 connection(s)
I20260808 07:01:57.141652 18982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:01:57.145038 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 18982
I20260808 07:01:57.145136 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:01:57.145395 14140 external_mini_cluster.cc:1428] Setting key ace6e1adcce882ea065cfb62e3bc0af6
I20260808 07:01:57.145668 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:57.142568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:47723 (local address 127.13.207.62:34353)
0808 07:01:57.142662 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:57.142664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:57.143075 (+ 411us) server_negotiation.cc:407] Connection header received
0808 07:01:57.143676 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:57.143677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:57.143684 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:57.143743 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:57.145342 (+ 1599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:57.145344 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:57.145344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:57.145350 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:57.145367 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:57.145372 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:57.145462 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:57.145571 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 07:01:57.145590 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 07:01:57.145926 19120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:57.142562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:34353 (local address 127.13.207.1:47723)
0808 07:01:57.142843 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:57.142902 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:57.143544 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:57.143829 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:57.143833 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:57.143851 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:57.145135 (+ 1284us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:01:57.145136 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:57.145186 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:01:57.145235 (+ 49us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:57.145459 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:57.145467 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:57.145470 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:01:57.145577 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 07:01:57.145656 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":153,"threads_started":1}
I20260808 07:01:57.146562 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:34353
--builtin_ntp_servers=127.13.207.20:44601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 07:01:57.146559 19118 heartbeater.cc:347] Connected to a master server at 127.13.207.62:34353
I20260808 07:01:57.146719 19118 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:57.146910 19118 heartbeater.cc:511] Master 127.13.207.62:34353 requested a full tablet report, sending...
I20260808 07:01:57.147475 18918 ts_manager.cc:194] Registered new tserver with Master: 56d93d4522f146f28193509788b9e179 (127.13.207.1:45963)
W20260808 07:01:57.236729 19121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:57.237004 19121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:57.237062 19121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:57.238745 19121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:57.238817 19121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:57.238850 19121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:01:57.240407 19121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:34353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:57.240684 19121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:57.240993 19121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:57.246896 19127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:57.247022 19126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:57.246999 19129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:57.247280 19121 server_base.cc:1034] running on GCE node
I20260808 07:01:57.247519 19121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:57.247754 19121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:57.248905 19121 hybrid_clock.cc:648] HybridClock initialized: now 1786172517248883 us; error 44 us; skew 500 ppm
I20260808 07:01:57.250279 19121 webserver.cc:460] Webserver started at http://127.13.207.2:39563/ using document root <none> and password file <none>
I20260808 07:01:57.250560 19121 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:57.250613 19121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:57.250792 19121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:57.251976 19121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "bad8cb7243b14ccf8f42f21ebd723739"
format_stamp: "Formatted at 2026-08-08 07:01:57 on dist-test-slave-8tdl"
server_key: "3fda63ccbf36008e57d03839e16e6465"
server_key_iv: "de7782b73e11e21f3dfcac961858745a"
server_key_version: "encryptionkey@0"
I20260808 07:01:57.252307 19121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "bad8cb7243b14ccf8f42f21ebd723739"
format_stamp: "Formatted at 2026-08-08 07:01:57 on dist-test-slave-8tdl"
server_key: "3fda63ccbf36008e57d03839e16e6465"
server_key_iv: "de7782b73e11e21f3dfcac961858745a"
server_key_version: "encryptionkey@0"
I20260808 07:01:57.253840 19121 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:01:57.254807 19135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:57.255056 19121 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:57.255159 19121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "bad8cb7243b14ccf8f42f21ebd723739"
format_stamp: "Formatted at 2026-08-08 07:01:57 on dist-test-slave-8tdl"
server_key: "3fda63ccbf36008e57d03839e16e6465"
server_key_iv: "de7782b73e11e21f3dfcac961858745a"
server_key_version: "encryptionkey@0"
I20260808 07:01:57.255246 19121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:57.259155 19121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:57.259508 19121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:57.259654 19121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:57.259898 19121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:57.260255 19121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:57.260322 19121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:57.260361 19121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:57.260388 19121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:57.266839 19121 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:41991
I20260808 07:01:57.267014 19252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:41991 every 8 connection(s)
I20260808 07:01:57.267352 19121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:01:57.270987 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:57.268311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:46633 (local address 127.13.207.62:34353)
0808 07:01:57.268397 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:57.268400 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:57.268650 (+ 250us) server_negotiation.cc:407] Connection header received
0808 07:01:57.269197 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:57.269198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:57.269206 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:57.269278 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:57.270718 (+ 1440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:57.270720 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:57.270720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:57.270726 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:57.270744 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:57.270748 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:57.270803 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:57.270882 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 07:01:57.270905 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 07:01:57.271224 19255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:57.268299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:34353 (local address 127.13.207.2:46633)
0808 07:01:57.268513 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:57.268557 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:57.269076 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:57.269357 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:57.269361 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:57.269372 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:57.270525 (+ 1153us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:01:57.270526 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:57.270559 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:01:57.270592 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:57.270799 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:57.270805 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:57.270809 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:01:57.270893 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 07:01:57.270983 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":102,"threads_started":1}
I20260808 07:01:57.271921 19253 heartbeater.cc:347] Connected to a master server at 127.13.207.62:34353
I20260808 07:01:57.272028 19253 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:57.272164 19253 heartbeater.cc:511] Master 127.13.207.62:34353 requested a full tablet report, sending...
I20260808 07:01:57.272560 18918 ts_manager.cc:194] Registered new tserver with Master: bad8cb7243b14ccf8f42f21ebd723739 (127.13.207.2:41991)
I20260808 07:01:57.273247 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 19121
I20260808 07:01:57.273366 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:01:57.273578 14140 external_mini_cluster.cc:1428] Setting key 15f049e6951c2aa47dfa1213cb444e4f
I20260808 07:01:57.275038 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:34353
--builtin_ntp_servers=127.13.207.20:44601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 07:01:57.370880 19256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:57.371196 19256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:57.371230 19256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:57.372761 19256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:57.372825 19256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:57.372857 19256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:01:57.374379 19256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:34353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:57.374622 19256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:57.375010 19256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:57.380671 19262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:57.380755 19256 server_base.cc:1034] running on GCE node
W20260808 07:01:57.380692 19266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:57.380697 19263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:57.381328 19256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:57.381700 19256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:57.382890 19256 hybrid_clock.cc:648] HybridClock initialized: now 1786172517382845 us; error 65 us; skew 500 ppm
I20260808 07:01:57.384910 19256 webserver.cc:460] Webserver started at http://127.13.207.3:39099/ using document root <none> and password file <none>
I20260808 07:01:57.385272 19256 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:57.385347 19256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:57.385505 19256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:57.387152 19256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "af124e9d78c74700b8656bd9c7d0457a"
format_stamp: "Formatted at 2026-08-08 07:01:57 on dist-test-slave-8tdl"
server_key: "95b1453fb3a53aa97182a90e2fc7ed72"
server_key_iv: "a57a599c500e59b74a72585028da69e1"
server_key_version: "encryptionkey@0"
I20260808 07:01:57.387641 19256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "af124e9d78c74700b8656bd9c7d0457a"
format_stamp: "Formatted at 2026-08-08 07:01:57 on dist-test-slave-8tdl"
server_key: "95b1453fb3a53aa97182a90e2fc7ed72"
server_key_iv: "a57a599c500e59b74a72585028da69e1"
server_key_version: "encryptionkey@0"
I20260808 07:01:57.389353 19256 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:01:57.390764 19272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:57.390987 19256 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:57.391199 19256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "af124e9d78c74700b8656bd9c7d0457a"
format_stamp: "Formatted at 2026-08-08 07:01:57 on dist-test-slave-8tdl"
server_key: "95b1453fb3a53aa97182a90e2fc7ed72"
server_key_iv: "a57a599c500e59b74a72585028da69e1"
server_key_version: "encryptionkey@0"
I20260808 07:01:57.391302 19256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:57.395406 19256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:57.395756 19256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:57.395926 19256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:57.396195 19256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:01:57.396605 19256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:01:57.396667 19256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:57.396715 19256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:01:57.396747 19256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:57.404179 19256 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:46689
I20260808 07:01:57.404281 19389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:46689 every 8 connection(s)
I20260808 07:01:57.404664 19256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:01:57.408812 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:57.405766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:33279 (local address 127.13.207.62:34353)
0808 07:01:57.405950 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:57.405952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:57.406252 (+ 300us) server_negotiation.cc:407] Connection header received
0808 07:01:57.406782 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:57.406784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:57.406791 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:57.406859 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:57.408409 (+ 1550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:57.408411 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:57.408411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:57.408417 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:57.408439 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:57.408445 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:57.408521 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:57.408726 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 07:01:57.408742 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260808 07:01:57.409053 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:57.405675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:34353 (local address 127.13.207.3:33279)
0808 07:01:57.406052 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:57.406090 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:57.406602 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:57.406929 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:57.406933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:57.406942 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:57.408182 (+ 1240us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:01:57.408183 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:57.408223 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:01:57.408261 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:57.408584 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:57.408591 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:57.408594 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:01:57.408680 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 07:01:57.408744 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":203,"threads_started":1}
I20260808 07:01:57.409797 19390 heartbeater.cc:347] Connected to a master server at 127.13.207.62:34353
I20260808 07:01:57.409981 19390 heartbeater.cc:464] Registering TS with master...
I20260808 07:01:57.410143 19390 heartbeater.cc:511] Master 127.13.207.62:34353 requested a full tablet report, sending...
I20260808 07:01:57.410634 18918 ts_manager.cc:194] Registered new tserver with Master: af124e9d78c74700b8656bd9c7d0457a (127.13.207.3:46689)
I20260808 07:01:57.411597 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 19256
I20260808 07:01:57.411689 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:01:57.411826 14140 external_mini_cluster.cc:1428] Setting key bf9b6f15998f10835ba8832405edc758
I20260808 07:01:57.413506 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:01:57.416952 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:57.415356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47610 (local address 127.13.207.62:34353)
0808 07:01:57.415527 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:57.415529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:57.415637 (+ 108us) server_negotiation.cc:407] Connection header received
0808 07:01:57.415710 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:57.415711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:57.415719 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:57.415786 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:57.416537 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:57.416539 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:57.416539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:57.416545 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:57.416569 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:57.416573 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:57.416737 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:57.416795 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 07:01:57.416814 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260808 07:01:57.419253 18918 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:01:57.421007 18918 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:01:57.426036 19403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:57.423966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51776 (local address 127.13.207.3:46689)
0808 07:01:57.424146 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:57.424152 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:57.424169 (+ 17us) server_negotiation.cc:407] Connection header received
0808 07:01:57.424242 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:57.424245 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:57.424254 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:57.424345 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:57.425579 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:57.425589 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:57.425589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:57.425594 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:57.425630 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:57.425650 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:57.425738 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:57.425911 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 07:01:57.425930 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":101,"threads_started":1}
I20260808 07:01:57.426018 19399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:57.423636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:41991 (local address 127.0.0.1:47906)
0808 07:01:57.423864 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:57.423910 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:57.424009 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:57.424704 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:57.424715 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:57.424722 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:57.425375 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:01:57.425375 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:57.425398 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:01:57.425430 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:57.425830 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:57.425831 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:57.425832 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:01:57.425908 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 07:01:57.425927 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":92,"threads_started":1}
I20260808 07:01:57.426147 19400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:57.423654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:46689 (local address 127.0.0.1:51776)
0808 07:01:57.424121 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:57.424132 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:57.424167 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:57.424704 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:57.424720 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:57.424722 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:57.425375 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:01:57.425375 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:57.425398 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:01:57.425430 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:57.425783 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:57.425793 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:57.425797 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:01:57.425888 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 07:01:57.425907 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":143,"threads_started":1}
I20260808 07:01:57.426407 19398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:57.423655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:45963 (local address 127.0.0.1:55260)
0808 07:01:57.424268 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:57.424277 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:57.424314 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:57.424857 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:57.424858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:57.424861 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:57.425921 (+ 1060us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:01:57.425922 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:57.425929 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:01:57.425942 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:57.426250 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:57.426251 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:57.426252 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:01:57.426318 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 07:01:57.426333 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"mutex_wait_us":2,"thread_start_us":107,"threads_started":1}
I20260808 07:01:57.426707 19404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:57.424053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55260 (local address 127.13.207.1:45963)
0808 07:01:57.424356 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:57.424361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:57.424372 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:01:57.424506 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:57.424511 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:57.424522 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:57.424592 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:57.426043 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:57.426056 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:57.426056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:57.426061 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:57.426104 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:57.426140 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:57.426218 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:57.426540 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 07:01:57.426561 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":99,"threads_started":1}
I20260808 07:01:57.426842 19402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:57.423739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47906 (local address 127.13.207.2:41991)
0808 07:01:57.424200 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:57.424204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:57.424213 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:01:57.424242 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:57.424244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:57.424254 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:57.424345 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:57.425549 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:57.425560 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:57.425561 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:57.425566 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:57.425614 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:57.425637 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:57.425848 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:57.426716 (+ 868us) server_negotiation.cc:299] Negotiation successful
0808 07:01:57.426750 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":101,"threads_started":1}
I20260808 07:01:57.427155 19321 tablet_service.cc:1467] Processing CreateTablet for tablet 7a734d5d9089423baf70d5a275d679ef (DEFAULT_TABLE table=test-table [id=3ceda33873074d6c91039726873ef076]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:57.427603 19321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a734d5d9089423baf70d5a275d679ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:57.427474 19046 tablet_service.cc:1467] Processing CreateTablet for tablet 7a734d5d9089423baf70d5a275d679ef (DEFAULT_TABLE table=test-table [id=3ceda33873074d6c91039726873ef076]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:57.427558 19184 tablet_service.cc:1467] Processing CreateTablet for tablet 7a734d5d9089423baf70d5a275d679ef (DEFAULT_TABLE table=test-table [id=3ceda33873074d6c91039726873ef076]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:01:57.427853 19046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a734d5d9089423baf70d5a275d679ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:57.427946 19184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a734d5d9089423baf70d5a275d679ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:57.431056 19405 tablet_bootstrap.cc:492] T 7a734d5d9089423baf70d5a275d679ef P bad8cb7243b14ccf8f42f21ebd723739: Bootstrap starting.
I20260808 07:01:57.431842 19406 tablet_bootstrap.cc:492] T 7a734d5d9089423baf70d5a275d679ef P af124e9d78c74700b8656bd9c7d0457a: Bootstrap starting.
I20260808 07:01:57.432006 19407 tablet_bootstrap.cc:492] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179: Bootstrap starting.
I20260808 07:01:57.432286 19405 tablet_bootstrap.cc:654] T 7a734d5d9089423baf70d5a275d679ef P bad8cb7243b14ccf8f42f21ebd723739: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:57.432680 19406 tablet_bootstrap.cc:654] T 7a734d5d9089423baf70d5a275d679ef P af124e9d78c74700b8656bd9c7d0457a: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:57.432669 19405 log.cc:824] T 7a734d5d9089423baf70d5a275d679ef P bad8cb7243b14ccf8f42f21ebd723739: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:57.432739 19407 tablet_bootstrap.cc:654] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:57.433063 19407 log.cc:824] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:57.433079 19406 log.cc:824] T 7a734d5d9089423baf70d5a275d679ef P af124e9d78c74700b8656bd9c7d0457a: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:57.433669 19405 tablet_bootstrap.cc:492] T 7a734d5d9089423baf70d5a275d679ef P bad8cb7243b14ccf8f42f21ebd723739: No bootstrap required, opened a new log
I20260808 07:01:57.433753 19405 ts_tablet_manager.cc:1397] T 7a734d5d9089423baf70d5a275d679ef P bad8cb7243b14ccf8f42f21ebd723739: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:01:57.433893 19406 tablet_bootstrap.cc:492] T 7a734d5d9089423baf70d5a275d679ef P af124e9d78c74700b8656bd9c7d0457a: No bootstrap required, opened a new log
I20260808 07:01:57.433908 19407 tablet_bootstrap.cc:492] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179: No bootstrap required, opened a new log
I20260808 07:01:57.433967 19406 ts_tablet_manager.cc:1397] T 7a734d5d9089423baf70d5a275d679ef P af124e9d78c74700b8656bd9c7d0457a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:01:57.433990 19407 ts_tablet_manager.cc:1397] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:01:57.436488 19407 raft_consensus.cc:348] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad8cb7243b14ccf8f42f21ebd723739" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41991 } } peers { permanent_uuid: "56d93d4522f146f28193509788b9e179" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45963 } } peers { permanent_uuid: "af124e9d78c74700b8656bd9c7d0457a" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46689 } }
I20260808 07:01:57.436690 19407 raft_consensus.cc:374] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:57.436741 19407 raft_consensus.cc:729] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56d93d4522f146f28193509788b9e179, State: Initialized, Role: FOLLOWER
I20260808 07:01:57.436784 19405 raft_consensus.cc:348] T 7a734d5d9089423baf70d5a275d679ef P bad8cb7243b14ccf8f42f21ebd723739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad8cb7243b14ccf8f42f21ebd723739" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41991 } } peers { permanent_uuid: "56d93d4522f146f28193509788b9e179" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45963 } } peers { permanent_uuid: "af124e9d78c74700b8656bd9c7d0457a" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46689 } }
I20260808 07:01:57.436772 19406 raft_consensus.cc:348] T 7a734d5d9089423baf70d5a275d679ef P af124e9d78c74700b8656bd9c7d0457a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad8cb7243b14ccf8f42f21ebd723739" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41991 } } peers { permanent_uuid: "56d93d4522f146f28193509788b9e179" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45963 } } peers { permanent_uuid: "af124e9d78c74700b8656bd9c7d0457a" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46689 } }
I20260808 07:01:57.436877 19407 consensus_queue.cc:260] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad8cb7243b14ccf8f42f21ebd723739" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41991 } } peers { permanent_uuid: "56d93d4522f146f28193509788b9e179" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45963 } } peers { permanent_uuid: "af124e9d78c74700b8656bd9c7d0457a" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46689 } }
I20260808 07:01:57.436955 19405 raft_consensus.cc:374] T 7a734d5d9089423baf70d5a275d679ef P bad8cb7243b14ccf8f42f21ebd723739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:57.436966 19406 raft_consensus.cc:374] T 7a734d5d9089423baf70d5a275d679ef P af124e9d78c74700b8656bd9c7d0457a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:57.437006 19405 raft_consensus.cc:729] T 7a734d5d9089423baf70d5a275d679ef P bad8cb7243b14ccf8f42f21ebd723739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad8cb7243b14ccf8f42f21ebd723739, State: Initialized, Role: FOLLOWER
I20260808 07:01:57.437006 19406 raft_consensus.cc:729] T 7a734d5d9089423baf70d5a275d679ef P af124e9d78c74700b8656bd9c7d0457a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af124e9d78c74700b8656bd9c7d0457a, State: Initialized, Role: FOLLOWER
I20260808 07:01:57.437148 19405 consensus_queue.cc:260] T 7a734d5d9089423baf70d5a275d679ef P bad8cb7243b14ccf8f42f21ebd723739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad8cb7243b14ccf8f42f21ebd723739" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41991 } } peers { permanent_uuid: "56d93d4522f146f28193509788b9e179" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45963 } } peers { permanent_uuid: "af124e9d78c74700b8656bd9c7d0457a" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46689 } }
I20260808 07:01:57.437148 19406 consensus_queue.cc:260] T 7a734d5d9089423baf70d5a275d679ef P af124e9d78c74700b8656bd9c7d0457a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad8cb7243b14ccf8f42f21ebd723739" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41991 } } peers { permanent_uuid: "56d93d4522f146f28193509788b9e179" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45963 } } peers { permanent_uuid: "af124e9d78c74700b8656bd9c7d0457a" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46689 } }
I20260808 07:01:57.437260 19407 ts_tablet_manager.cc:1428] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260808 07:01:57.437407 19118 heartbeater.cc:503] Master 127.13.207.62:34353 was elected leader, sending a full tablet report...
I20260808 07:01:57.437434 19406 ts_tablet_manager.cc:1428] T 7a734d5d9089423baf70d5a275d679ef P af124e9d78c74700b8656bd9c7d0457a: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 07:01:57.437434 19405 ts_tablet_manager.cc:1428] T 7a734d5d9089423baf70d5a275d679ef P bad8cb7243b14ccf8f42f21ebd723739: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260808 07:01:57.437552 19253 heartbeater.cc:503] Master 127.13.207.62:34353 was elected leader, sending a full tablet report...
I20260808 07:01:57.437557 19390 heartbeater.cc:503] Master 127.13.207.62:34353 was elected leader, sending a full tablet report...
W20260808 07:01:57.518185 19254 tablet.cc:2369] T 7a734d5d9089423baf70d5a275d679ef P bad8cb7243b14ccf8f42f21ebd723739: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:01:57.642557 19119 tablet.cc:2369] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:01:57.655344 19391 tablet.cc:2369] T 7a734d5d9089423baf70d5a275d679ef P af124e9d78c74700b8656bd9c7d0457a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:01:57.826215 19411 raft_consensus.cc:482] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:01:57.826474 19411 raft_consensus.cc:504] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad8cb7243b14ccf8f42f21ebd723739" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41991 } } peers { permanent_uuid: "56d93d4522f146f28193509788b9e179" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45963 } } peers { permanent_uuid: "af124e9d78c74700b8656bd9c7d0457a" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46689 } }
I20260808 07:01:57.827013 19411 leader_election.cc:288] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad8cb7243b14ccf8f42f21ebd723739 (127.13.207.2:41991), af124e9d78c74700b8656bd9c7d0457a (127.13.207.3:46689)
I20260808 07:01:57.828948 19403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:57.827373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:33043 (local address 127.13.207.3:46689)
0808 07:01:57.827422 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:57.827424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:57.827594 (+ 170us) server_negotiation.cc:407] Connection header received
0808 07:01:57.827634 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:57.827635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:57.827642 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:57.827720 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:57.828617 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:57.828618 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:57.828618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:57.828624 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:57.828644 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:57.828650 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:57.828720 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:57.828854 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 07:01:57.828870 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:01:57.829012 19402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:57.827327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:59495 (local address 127.13.207.2:41991)
0808 07:01:57.827398 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:57.827401 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:57.827589 (+ 188us) server_negotiation.cc:407] Connection header received
0808 07:01:57.827634 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:57.827635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:57.827643 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:57.827720 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:57.828617 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:57.828618 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:57.828619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:57.828624 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:57.828644 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:57.828649 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:57.828725 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:57.828852 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 07:01:57.828870 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:01:57.829100 19415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:57.827230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:41991 (local address 127.13.207.1:59495)
0808 07:01:57.827435 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:57.827449 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:57.827541 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:57.827803 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:57.827804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:57.827805 (+ 1us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:57.828459 (+ 654us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:01:57.828460 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:57.828471 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:01:57.828488 (+ 17us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:57.828759 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:57.828760 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:57.828761 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:01:57.828845 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 07:01:57.828891 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":27,"thread_start_us":74,"threads_started":1}
I20260808 07:01:57.829197 19414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:57.827270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:46689 (local address 127.13.207.1:33043)
0808 07:01:57.827451 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:57.827460 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:57.827554 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:57.827784 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:57.827785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:57.827790 (+ 5us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:01:57.828493 (+ 703us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:01:57.828493 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:01:57.828497 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:01:57.828506 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:01:57.828759 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:01:57.828760 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:01:57.828761 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:01:57.828844 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 07:01:57.828864 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":78,"threads_started":1}
I20260808 07:01:57.829566 19342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a734d5d9089423baf70d5a275d679ef" candidate_uuid: "56d93d4522f146f28193509788b9e179" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af124e9d78c74700b8656bd9c7d0457a" is_pre_election: true
I20260808 07:01:57.829553 19205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a734d5d9089423baf70d5a275d679ef" candidate_uuid: "56d93d4522f146f28193509788b9e179" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad8cb7243b14ccf8f42f21ebd723739" is_pre_election: true
I20260808 07:01:57.829716 19342 raft_consensus.cc:2454] T 7a734d5d9089423baf70d5a275d679ef P af124e9d78c74700b8656bd9c7d0457a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56d93d4522f146f28193509788b9e179 in term 0.
I20260808 07:01:57.829742 19205 raft_consensus.cc:2454] T 7a734d5d9089423baf70d5a275d679ef P bad8cb7243b14ccf8f42f21ebd723739 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56d93d4522f146f28193509788b9e179 in term 0.
I20260808 07:01:57.830053 18997 leader_election.cc:302] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56d93d4522f146f28193509788b9e179, bad8cb7243b14ccf8f42f21ebd723739; no voters:
I20260808 07:01:57.830268 19411 raft_consensus.cc:2781] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:01:57.830363 19411 raft_consensus.cc:482] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:01:57.830488 19411 raft_consensus.cc:3037] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:57.831454 19411 raft_consensus.cc:504] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad8cb7243b14ccf8f42f21ebd723739" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41991 } } peers { permanent_uuid: "56d93d4522f146f28193509788b9e179" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45963 } } peers { permanent_uuid: "af124e9d78c74700b8656bd9c7d0457a" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46689 } }
I20260808 07:01:57.831650 19411 leader_election.cc:288] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179 [CANDIDATE]: Term 1 election: Requested vote from peers bad8cb7243b14ccf8f42f21ebd723739 (127.13.207.2:41991), af124e9d78c74700b8656bd9c7d0457a (127.13.207.3:46689)
I20260808 07:01:57.831861 19205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a734d5d9089423baf70d5a275d679ef" candidate_uuid: "56d93d4522f146f28193509788b9e179" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad8cb7243b14ccf8f42f21ebd723739"
I20260808 07:01:57.831892 19342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a734d5d9089423baf70d5a275d679ef" candidate_uuid: "56d93d4522f146f28193509788b9e179" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af124e9d78c74700b8656bd9c7d0457a"
I20260808 07:01:57.831979 19342 raft_consensus.cc:3037] T 7a734d5d9089423baf70d5a275d679ef P af124e9d78c74700b8656bd9c7d0457a [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:57.831979 19205 raft_consensus.cc:3037] T 7a734d5d9089423baf70d5a275d679ef P bad8cb7243b14ccf8f42f21ebd723739 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:57.832980 19342 raft_consensus.cc:2454] T 7a734d5d9089423baf70d5a275d679ef P af124e9d78c74700b8656bd9c7d0457a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56d93d4522f146f28193509788b9e179 in term 1.
I20260808 07:01:57.833065 19205 raft_consensus.cc:2454] T 7a734d5d9089423baf70d5a275d679ef P bad8cb7243b14ccf8f42f21ebd723739 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56d93d4522f146f28193509788b9e179 in term 1.
I20260808 07:01:57.833207 18999 leader_election.cc:302] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56d93d4522f146f28193509788b9e179, af124e9d78c74700b8656bd9c7d0457a; no voters:
I20260808 07:01:57.833349 19411 raft_consensus.cc:2781] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:57.833601 19411 raft_consensus.cc:686] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179 [term 1 LEADER]: Becoming Leader. State: Replica: 56d93d4522f146f28193509788b9e179, State: Running, Role: LEADER
I20260808 07:01:57.833849 19411 consensus_queue.cc:237] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad8cb7243b14ccf8f42f21ebd723739" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41991 } } peers { permanent_uuid: "56d93d4522f146f28193509788b9e179" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45963 } } peers { permanent_uuid: "af124e9d78c74700b8656bd9c7d0457a" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46689 } }
I20260808 07:01:57.835049 18916 catalog_manager.cc:5495] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56d93d4522f146f28193509788b9e179 (127.13.207.1). New cstate: current_term: 1 leader_uuid: "56d93d4522f146f28193509788b9e179" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad8cb7243b14ccf8f42f21ebd723739" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56d93d4522f146f28193509788b9e179" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af124e9d78c74700b8656bd9c7d0457a" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46689 } health_report { overall_health: UNKNOWN } } }
I20260808 07:01:57.871457 19404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:57.870005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55264 (local address 127.13.207.1:45963)
0808 07:01:57.870131 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:57.870134 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:57.870145 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:01:57.870193 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:57.870194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:57.870210 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:57.870277 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:57.871175 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:57.871176 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:57.871176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:57.871180 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:57.871202 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:57.871207 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:57.871261 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:57.871369 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 07:01:57.871383 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260808 07:01:57.873818 19342 raft_consensus.cc:1261] T 7a734d5d9089423baf70d5a275d679ef P af124e9d78c74700b8656bd9c7d0457a [term 1 FOLLOWER]: Refusing update from remote peer 56d93d4522f146f28193509788b9e179: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:01:57.873818 19205 raft_consensus.cc:1261] T 7a734d5d9089423baf70d5a275d679ef P bad8cb7243b14ccf8f42f21ebd723739 [term 1 FOLLOWER]: Refusing update from remote peer 56d93d4522f146f28193509788b9e179: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:01:57.874323 19411 consensus_queue.cc:1035] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af124e9d78c74700b8656bd9c7d0457a" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:01:57.874514 19416 consensus_queue.cc:1035] T 7a734d5d9089423baf70d5a275d679ef P 56d93d4522f146f28193509788b9e179 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad8cb7243b14ccf8f42f21ebd723739" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:01:57.877068 19419 mvcc.cc:205] Tried to move back new op lower bound from 7316162633207627776 to 7316162633048223744. Current Snapshot: MvccSnapshot[applied={T|T < 7316162633207627776}]
I20260808 07:01:57.877408 19420 mvcc.cc:205] Tried to move back new op lower bound from 7316162633207627776 to 7316162633048223744. Current Snapshot: MvccSnapshot[applied={T|T < 7316162633207627776}]
I20260808 07:01:57.877797 19421 mvcc.cc:205] Tried to move back new op lower bound from 7316162633207627776 to 7316162633048223744. Current Snapshot: MvccSnapshot[applied={T|T < 7316162633207627776}]
I20260808 07:01:57.880792 18916 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:01:57.880919 18916 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:01:57.881824 18916 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d008f8157bf94b05a8a8a8c1d755fd0c: Sending DeleteTablet for 3 replicas of tablet 7a734d5d9089423baf70d5a275d679ef
I20260808 07:01:57.882303 19184 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a734d5d9089423baf70d5a275d679ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:57 UTC) from {username='slave'} at 127.0.0.1:47906
I20260808 07:01:57.882293 19321 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a734d5d9089423baf70d5a275d679ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:57 UTC) from {username='slave'} at 127.0.0.1:51776
I20260808 07:01:57.882861 19046 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a734d5d9089423baf70d5a275d679ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:01:57 UTC) from {username='slave'} at 127.0.0.1:55260
I20260808 07:01:57.882944 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 18982
I20260808 07:01:57.886976 19429 tablet_replica.cc:330] T 7a734d5d9089423baf70d5a275d679ef P bad8cb7243b14ccf8f42f21ebd723739: stopping tablet replica
I20260808 07:01:57.887166 19429 raft_consensus.cc:2229] T 7a734d5d9089423baf70d5a275d679ef P bad8cb7243b14ccf8f42f21ebd723739 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:01:57.887347 19429 raft_consensus.cc:2258] T 7a734d5d9089423baf70d5a275d679ef P bad8cb7243b14ccf8f42f21ebd723739 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:01:57.887588 19430 tablet_replica.cc:330] T 7a734d5d9089423baf70d5a275d679ef P af124e9d78c74700b8656bd9c7d0457a: stopping tablet replica
I20260808 07:01:57.887712 19430 raft_consensus.cc:2229] T 7a734d5d9089423baf70d5a275d679ef P af124e9d78c74700b8656bd9c7d0457a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:01:57.887818 19430 raft_consensus.cc:2258] T 7a734d5d9089423baf70d5a275d679ef P af124e9d78c74700b8656bd9c7d0457a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:01:57.888024 19429 ts_tablet_manager.cc:1905] T 7a734d5d9089423baf70d5a275d679ef P bad8cb7243b14ccf8f42f21ebd723739: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:01:57.888286 19430 ts_tablet_manager.cc:1905] T 7a734d5d9089423baf70d5a275d679ef P af124e9d78c74700b8656bd9c7d0457a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:01:57.890360 19429 ts_tablet_manager.cc:1918] T 7a734d5d9089423baf70d5a275d679ef P bad8cb7243b14ccf8f42f21ebd723739: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:01:57.890480 19429 log.cc:1196] T 7a734d5d9089423baf70d5a275d679ef P bad8cb7243b14ccf8f42f21ebd723739: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-1/wal/wals/7a734d5d9089423baf70d5a275d679ef
I20260808 07:01:57.890841 19429 ts_tablet_manager.cc:1939] T 7a734d5d9089423baf70d5a275d679ef P bad8cb7243b14ccf8f42f21ebd723739: Deleting consensus metadata
I20260808 07:01:57.891405 19430 ts_tablet_manager.cc:1918] T 7a734d5d9089423baf70d5a275d679ef P af124e9d78c74700b8656bd9c7d0457a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:01:57.891666 18902 catalog_manager.cc:4841] TS bad8cb7243b14ccf8f42f21ebd723739 (127.13.207.2:41991): tablet 7a734d5d9089423baf70d5a275d679ef (table test-table [id=3ceda33873074d6c91039726873ef076]) successfully deleted
I20260808 07:01:57.891690 19430 log.cc:1196] T 7a734d5d9089423baf70d5a275d679ef P af124e9d78c74700b8656bd9c7d0457a: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/7a734d5d9089423baf70d5a275d679ef
W20260808 07:01:57.891753 18905 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.207.1:45963 (error 108)
I20260808 07:01:57.892166 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 19121
I20260808 07:01:57.892201 19430 ts_tablet_manager.cc:1939] T 7a734d5d9089423baf70d5a275d679ef P af124e9d78c74700b8656bd9c7d0457a: Deleting consensus metadata
I20260808 07:01:57.892382 19400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:57.892216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:45963 (local address 127.0.0.1:55266)
0808 07:01:57.892284 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:57.892331 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.1:45963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 07:01:57.896041 18905 catalog_manager.cc:4584] TS 56d93d4522f146f28193509788b9e179 (127.13.207.1:45963): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7a734d5d9089423baf70d5a275d679ef: Network error: Client connection negotiation failed: client connection to 127.13.207.1:45963: connect: Connection refused (error 111)
I20260808 07:01:57.902714 18904 catalog_manager.cc:4841] TS af124e9d78c74700b8656bd9c7d0457a (127.13.207.3:46689): tablet 7a734d5d9089423baf70d5a275d679ef (table test-table [id=3ceda33873074d6c91039726873ef076]) successfully deleted
I20260808 07:01:57.902961 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 19256
I20260808 07:01:57.908641 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 18887
2026-08-08T07:01:57Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1102 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 07:01:57.930451 14140 mini_oidc.cc:152] Starting JWKS server
I20260808 07:01:57.932073 14140 webserver.cc:460] Webserver started at https://127.0.0.1:38255/ using document root <none> and password file <none>
I20260808 07:01:57.932179 14140 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 07:01:57.932348 14140 webserver.cc:460] Webserver started at http://127.0.0.1:40889/ using document root <none> and password file <none>
2026-08-08T07:01:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:01:57Z Disabled control of system clock
I20260808 07:01:57.943645 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:41207
--webserver_interface=127.13.207.62
--webserver_port=0
--builtin_ntp_servers=127.13.207.20:43261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:41207
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38255/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172493058068-14140-0/testchainca.cert with env {}
W20260808 07:01:58.032016 19439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:58.032269 19439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:58.032296 19439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:58.033576 19439 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:01:58.033625 19439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:58.033639 19439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:58.033653 19439 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:01:58.033665 19439 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 07:01:58.033679 19439 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 07:01:58.033690 19439 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38255/jwks.json
I20260808 07:01:58.035531 19439 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:43261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:41207
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:41207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38255/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172493058068-14140-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:58.035778 19439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:58.035959 19439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:58.041764 19444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:58.041694 19445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:58.041694 19447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:58.041814 19439 server_base.cc:1034] running on GCE node
I20260808 07:01:58.042397 19439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:58.042861 19439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:58.044104 19439 hybrid_clock.cc:648] HybridClock initialized: now 1786172518044080 us; error 50 us; skew 500 ppm
I20260808 07:01:58.045696 19439 webserver.cc:460] Webserver started at http://127.13.207.62:39919/ using document root <none> and password file <none>
I20260808 07:01:58.046049 19439 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:58.046140 19439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:58.046305 19439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:58.047610 19439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "ffb275ecaf35444aa520110580c2c0a5"
format_stamp: "Formatted at 2026-08-08 07:01:58 on dist-test-slave-8tdl"
server_key: "4e06f04d8b70461575f6f2af0f377038"
server_key_iv: "837a2ad2d6272dbfe436609deb59fab2"
server_key_version: "encryptionkey@0"
I20260808 07:01:58.048030 19439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "ffb275ecaf35444aa520110580c2c0a5"
format_stamp: "Formatted at 2026-08-08 07:01:58 on dist-test-slave-8tdl"
server_key: "4e06f04d8b70461575f6f2af0f377038"
server_key_iv: "837a2ad2d6272dbfe436609deb59fab2"
server_key_version: "encryptionkey@0"
I20260808 07:01:58.049678 19439 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 07:01:58.050798 19453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:58.051196 19439 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:58.051438 19439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "ffb275ecaf35444aa520110580c2c0a5"
format_stamp: "Formatted at 2026-08-08 07:01:58 on dist-test-slave-8tdl"
server_key: "4e06f04d8b70461575f6f2af0f377038"
server_key_iv: "837a2ad2d6272dbfe436609deb59fab2"
server_key_version: "encryptionkey@0"
I20260808 07:01:58.051537 19439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:58.066233 19439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:58.066781 19439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:58.066985 19439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:58.072865 19439 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:41207
I20260808 07:01:58.072955 19507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:41207 every 8 connection(s)
I20260808 07:01:58.073462 19439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:01:58.074671 19508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:58.077324 19508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffb275ecaf35444aa520110580c2c0a5: Bootstrap starting.
I20260808 07:01:58.078012 19508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffb275ecaf35444aa520110580c2c0a5: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:58.078289 19508 log.cc:824] T 00000000000000000000000000000000 P ffb275ecaf35444aa520110580c2c0a5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:58.079305 19508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffb275ecaf35444aa520110580c2c0a5: No bootstrap required, opened a new log
I20260808 07:01:58.079838 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 19439
I20260808 07:01:58.079959 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:01:58.080171 14140 external_mini_cluster.cc:1428] Setting key 642cda67a15a6c3f5fdcd885251d5a12
I20260808 07:01:58.081643 19508 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffb275ecaf35444aa520110580c2c0a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb275ecaf35444aa520110580c2c0a5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41207 } }
I20260808 07:01:58.081840 19508 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffb275ecaf35444aa520110580c2c0a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:58.081889 19508 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffb275ecaf35444aa520110580c2c0a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffb275ecaf35444aa520110580c2c0a5, State: Initialized, Role: FOLLOWER
I20260808 07:01:58.082053 19508 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffb275ecaf35444aa520110580c2c0a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb275ecaf35444aa520110580c2c0a5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41207 } }
I20260808 07:01:58.082147 19508 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffb275ecaf35444aa520110580c2c0a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:01:58.082195 19508 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffb275ecaf35444aa520110580c2c0a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:01:58.082245 19508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffb275ecaf35444aa520110580c2c0a5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:58.083076 19508 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffb275ecaf35444aa520110580c2c0a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb275ecaf35444aa520110580c2c0a5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41207 } }
I20260808 07:01:58.083210 19508 leader_election.cc:302] T 00000000000000000000000000000000 P ffb275ecaf35444aa520110580c2c0a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffb275ecaf35444aa520110580c2c0a5; no voters:
I20260808 07:01:58.083413 19508 leader_election.cc:288] T 00000000000000000000000000000000 P ffb275ecaf35444aa520110580c2c0a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:01:58.083479 19513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffb275ecaf35444aa520110580c2c0a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:58.083640 19513 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffb275ecaf35444aa520110580c2c0a5 [term 1 LEADER]: Becoming Leader. State: Replica: ffb275ecaf35444aa520110580c2c0a5, State: Running, Role: LEADER
I20260808 07:01:58.083734 19508 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffb275ecaf35444aa520110580c2c0a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:01:58.083739 19513 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffb275ecaf35444aa520110580c2c0a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb275ecaf35444aa520110580c2c0a5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41207 } }
I20260808 07:01:58.084164 19513 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffb275ecaf35444aa520110580c2c0a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffb275ecaf35444aa520110580c2c0a5. Latest consensus state: current_term: 1 leader_uuid: "ffb275ecaf35444aa520110580c2c0a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb275ecaf35444aa520110580c2c0a5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41207 } } }
I20260808 07:01:58.084257 19513 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffb275ecaf35444aa520110580c2c0a5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:58.084182 19514 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffb275ecaf35444aa520110580c2c0a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffb275ecaf35444aa520110580c2c0a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb275ecaf35444aa520110580c2c0a5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41207 } } }
I20260808 07:01:58.084491 19520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:01:58.085153 19520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:01:58.084479 19514 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffb275ecaf35444aa520110580c2c0a5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:58.087112 19520 catalog_manager.cc:1347] Generated new cluster ID: 0ce3cfc0f3904c118a9ec8e45f53b95c
I20260808 07:01:58.087179 19520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:01:58.087738 19512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:58.080769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48520 (local address 127.13.207.62:41207)
0808 07:01:58.081244 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:58.081252 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:58.081280 (+ 28us) server_negotiation.cc:407] Connection header received
0808 07:01:58.081636 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:58.081645 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:58.081717 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:58.081900 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:58.082142 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.083762 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.085006 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.085817 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.086331 (+ 514us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:58.086864 (+ 533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:58.086868 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:58.086868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:58.087095 (+ 227us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:58.087141 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:58.087157 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:58.087220 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:58.087420 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 07:01:58.087509 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":350,"threads_started":1}
I20260808 07:01:58.088760 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:01:58.105057 19520 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:01:58.105816 19520 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:01:58.113364 19520 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffb275ecaf35444aa520110580c2c0a5: Generated new TSK 0
I20260808 07:01:58.113620 19520 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 07:01:58.138188 19538 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 07:01:58.138346 19538 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260808 07:01:58.139053 19537 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:58.120769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:41207 (local address 127.0.0.1:48532)
0808 07:01:58.120979 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:58.120991 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:58.121058 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:58.121297 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:58.121297 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:58.121300 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0808 07:01:58.121370 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:58.121371 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.122551 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.122553 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:58.123289 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:58.123292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.123864 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.123865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:58.123924 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:58.123951 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 07:01:58.138458 (+ 14507us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38255/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 07:01:58.138575 (+ 117us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.207.62:41207: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38255/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":107,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.13.207.62:41207: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38255/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 07:01:58.139412 19512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:58.120872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48532 (local address 127.13.207.62:41207)
0808 07:01:58.120938 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:58.120941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:58.121059 (+ 118us) server_negotiation.cc:407] Connection header received
0808 07:01:58.121157 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:58.121158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:58.121173 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:58.121264 (+ 91us) server_negotiation.cc:226] Negotiated authn=JWT
0808 07:01:58.121446 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.122222 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.123398 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.123728 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.123819 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:58.124099 (+ 280us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 07:01:58.138259 (+ 14160us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38255/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 07:01:58.139345 (+ 1086us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48532: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38255/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":32}
W20260808 07:01:58.139470 19512 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48532: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38255/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
I20260808 07:01:58.139498 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 19439
2026-08-08T07:01:58Z chronyd exiting
I20260808 07:01:58.178296 14140 test_util.cc:175] -----------------------------------------------
I20260808 07:01:58.178433 14140 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172493058068-14140-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (249 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 07:01:58.179577 14140 mini_oidc.cc:152] Starting JWKS server
I20260808 07:01:58.180378 14140 webserver.cc:460] Webserver started at https://127.0.0.1:46111/ using document root <none> and password file <none>
I20260808 07:01:58.180469 14140 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 07:01:58.180672 14140 webserver.cc:460] Webserver started at http://127.0.0.1:42007/ using document root <none> and password file <none>
2026-08-08T07:01:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:01:58Z Disabled control of system clock
I20260808 07:01:58.191632 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:42173
--webserver_interface=127.13.207.62
--webserver_port=0
--builtin_ntp_servers=127.13.207.20:34739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:42173
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46111/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172493058068-14140-0/testchainca.cert with env {}
W20260808 07:01:58.291071 19545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:58.291332 19545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:58.291368 19545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:58.292822 19545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:01:58.292892 19545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:58.292908 19545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:58.292923 19545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:01:58.292937 19545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 07:01:58.292953 19545 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 07:01:58.292965 19545 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46111/jwks.json
I20260808 07:01:58.294868 19545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:34739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:42173
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:42173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46111/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172493058068-14140-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:58.295117 19545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:58.295325 19545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:58.301658 19551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:58.301731 19550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:58.301662 19545 server_base.cc:1034] running on GCE node
W20260808 07:01:58.301757 19553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:58.302225 19545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:58.302629 19545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:58.303839 19545 hybrid_clock.cc:648] HybridClock initialized: now 1786172518303806 us; error 48 us; skew 500 ppm
I20260808 07:01:58.305414 19545 webserver.cc:460] Webserver started at http://127.13.207.62:43633/ using document root <none> and password file <none>
I20260808 07:01:58.305745 19545 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:58.305799 19545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:58.305946 19545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:58.307262 19545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "a6913aadddfb4d4db90578b0f7b50235"
format_stamp: "Formatted at 2026-08-08 07:01:58 on dist-test-slave-8tdl"
server_key: "6fa837d3f156469c8d4a7dd9a8e43038"
server_key_iv: "d91abdcaff9d269843e6e456e1353c13"
server_key_version: "encryptionkey@0"
I20260808 07:01:58.307704 19545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "a6913aadddfb4d4db90578b0f7b50235"
format_stamp: "Formatted at 2026-08-08 07:01:58 on dist-test-slave-8tdl"
server_key: "6fa837d3f156469c8d4a7dd9a8e43038"
server_key_iv: "d91abdcaff9d269843e6e456e1353c13"
server_key_version: "encryptionkey@0"
I20260808 07:01:58.309455 19545 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:01:58.310479 19559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:58.310866 19545 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:58.310973 19545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "a6913aadddfb4d4db90578b0f7b50235"
format_stamp: "Formatted at 2026-08-08 07:01:58 on dist-test-slave-8tdl"
server_key: "6fa837d3f156469c8d4a7dd9a8e43038"
server_key_iv: "d91abdcaff9d269843e6e456e1353c13"
server_key_version: "encryptionkey@0"
I20260808 07:01:58.311112 19545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:58.325966 19545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:58.326413 19545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:58.326614 19545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:58.331477 19545 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:42173
I20260808 07:01:58.331522 19612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:42173 every 8 connection(s)
I20260808 07:01:58.331938 19545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:01:58.332700 19613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:58.335820 19613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235: Bootstrap starting.
I20260808 07:01:58.336488 19613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:58.336745 19613 log.cc:824] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:58.337658 19613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235: No bootstrap required, opened a new log
I20260808 07:01:58.338210 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 19545
I20260808 07:01:58.338326 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:01:58.338551 14140 external_mini_cluster.cc:1428] Setting key 45821df9db7c6cb6a76057f382ce1a12
I20260808 07:01:58.340157 19613 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6913aadddfb4d4db90578b0f7b50235" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 42173 } }
I20260808 07:01:58.340359 19613 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:58.340412 19613 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6913aadddfb4d4db90578b0f7b50235, State: Initialized, Role: FOLLOWER
I20260808 07:01:58.340565 19613 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6913aadddfb4d4db90578b0f7b50235" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 42173 } }
I20260808 07:01:58.340664 19613 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:01:58.340715 19613 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:01:58.340795 19613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:58.341497 19613 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6913aadddfb4d4db90578b0f7b50235" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 42173 } }
I20260808 07:01:58.341635 19613 leader_election.cc:302] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6913aadddfb4d4db90578b0f7b50235; no voters:
I20260808 07:01:58.341840 19613 leader_election.cc:288] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:01:58.341889 19618 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:58.342043 19618 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235 [term 1 LEADER]: Becoming Leader. State: Replica: a6913aadddfb4d4db90578b0f7b50235, State: Running, Role: LEADER
I20260808 07:01:58.342131 19618 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6913aadddfb4d4db90578b0f7b50235" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 42173 } }
I20260808 07:01:58.342170 19613 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:01:58.342473 19618 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6913aadddfb4d4db90578b0f7b50235. Latest consensus state: current_term: 1 leader_uuid: "a6913aadddfb4d4db90578b0f7b50235" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6913aadddfb4d4db90578b0f7b50235" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 42173 } } }
I20260808 07:01:58.342640 19618 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235 [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:58.342474 19620 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6913aadddfb4d4db90578b0f7b50235" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6913aadddfb4d4db90578b0f7b50235" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 42173 } } }
I20260808 07:01:58.343086 19620 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235 [sys.catalog]: This master's current role is: LEADER
W20260808 07:01:58.344482 19634 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:01:58.344551 19634 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:01:58.344611 19625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:01:58.344827 19625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:01:58.346153 19617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:58.339169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47936 (local address 127.13.207.62:42173)
0808 07:01:58.339439 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:58.339448 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:58.339493 (+ 45us) server_negotiation.cc:407] Connection header received
0808 07:01:58.340139 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:58.340146 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:58.340227 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:58.340378 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:58.340676 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.342383 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.343589 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.344238 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.344364 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:58.345171 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:58.345175 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:58.345175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:58.345387 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:58.345421 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:58.345452 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:58.345777 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:58.345811 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 07:01:58.345908 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":110,"threads_started":1}
I20260808 07:01:58.346627 19625 catalog_manager.cc:1347] Generated new cluster ID: de0b23228b8145c5bdc7735328e51358
I20260808 07:01:58.346714 19625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:01:58.347299 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:01:58.356832 19625 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:01:58.357645 19625 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:01:58.366230 19625 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6913aadddfb4d4db90578b0f7b50235: Generated new TSK 0
I20260808 07:01:58.366478 19625 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 07:01:58.392232 19644 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 07:01:58.392335 19644 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 07:01:58.392750 19617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:58.375573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47944 (local address 127.13.207.62:42173)
0808 07:01:58.375658 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:58.375661 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:58.375734 (+ 73us) server_negotiation.cc:407] Connection header received
0808 07:01:58.375816 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:58.375818 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:58.375832 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:58.375917 (+ 85us) server_negotiation.cc:226] Negotiated authn=JWT
0808 07:01:58.376120 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.376946 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.377968 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.378278 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.378366 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:58.378570 (+ 204us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 07:01:58.392484 (+ 13914us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46111/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 07:01:58.392669 (+ 185us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47944: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46111/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 07:01:58.392814 19617 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47944: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46111/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 07:01:58.392882 19643 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:58.375489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:42173 (local address 127.0.0.1:47944)
0808 07:01:58.375650 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:58.375661 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:58.375729 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:58.375923 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:58.375923 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:58.375925 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 07:01:58.376041 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:58.376042 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.377070 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.377071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:58.377867 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:58.377871 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.378385 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.378386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:58.378436 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:58.378442 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 07:01:58.392624 (+ 14182us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46111/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 07:01:58.392743 (+ 119us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.207.62:42173: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46111/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":108,"threads_started":1}
I20260808 07:01:58.393527 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 19545
2026-08-08T07:01:58Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (254 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 07:01:58.434285 14140 mini_oidc.cc:152] Starting JWKS server
I20260808 07:01:58.435125 14140 webserver.cc:460] Webserver started at https://127.0.0.1:43471/ using document root <none> and password file <none>
I20260808 07:01:58.435204 14140 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 07:01:58.435390 14140 webserver.cc:460] Webserver started at http://127.0.0.1:44429/ using document root <none> and password file <none>
2026-08-08T07:01:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:01:58Z Disabled control of system clock
I20260808 07:01:58.445704 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:34207
--webserver_interface=127.13.207.62
--webserver_port=0
--builtin_ntp_servers=127.13.207.20:34089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:34207
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43471/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 07:01:58.533772 19650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:58.534026 19650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:58.534052 19650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:58.535374 19650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:01:58.535427 19650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:58.535441 19650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:58.535454 19650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:01:58.535467 19650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 07:01:58.535480 19650 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 07:01:58.535491 19650 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43471/jwks.json
I20260808 07:01:58.536906 19650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:34089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:34207
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:34207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43471/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:58.537143 19650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:58.537406 19650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:58.543087 19660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:58.543097 19656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:58.543092 19657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:58.543605 19650 server_base.cc:1034] running on GCE node
I20260808 07:01:58.543867 19650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:58.544173 19650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:58.545356 19650 hybrid_clock.cc:648] HybridClock initialized: now 1786172518545330 us; error 44 us; skew 500 ppm
I20260808 07:01:58.547222 19650 webserver.cc:460] Webserver started at http://127.13.207.62:35915/ using document root <none> and password file <none>
I20260808 07:01:58.547528 19650 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:58.547587 19650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:58.547742 19650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:58.549024 19650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "440913078e1044f69dcc499818d4958b"
format_stamp: "Formatted at 2026-08-08 07:01:58 on dist-test-slave-8tdl"
server_key: "d8971a9c2beb41aa8b722e48092dca74"
server_key_iv: "d37edb40ea268a7388750986c1ceb081"
server_key_version: "encryptionkey@0"
I20260808 07:01:58.549507 19650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "440913078e1044f69dcc499818d4958b"
format_stamp: "Formatted at 2026-08-08 07:01:58 on dist-test-slave-8tdl"
server_key: "d8971a9c2beb41aa8b722e48092dca74"
server_key_iv: "d37edb40ea268a7388750986c1ceb081"
server_key_version: "encryptionkey@0"
I20260808 07:01:58.551319 19650 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260808 07:01:58.552331 19667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:58.552560 19650 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:58.552678 19650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "440913078e1044f69dcc499818d4958b"
format_stamp: "Formatted at 2026-08-08 07:01:58 on dist-test-slave-8tdl"
server_key: "d8971a9c2beb41aa8b722e48092dca74"
server_key_iv: "d37edb40ea268a7388750986c1ceb081"
server_key_version: "encryptionkey@0"
I20260808 07:01:58.552749 19650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:58.583490 19650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:58.583851 19650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:58.583973 19650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:58.588596 19650 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:34207
I20260808 07:01:58.588675 19721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:34207 every 8 connection(s)
I20260808 07:01:58.589057 19650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:01:58.589902 19722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:58.592008 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 19650
I20260808 07:01:58.592109 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:01:58.592294 14140 external_mini_cluster.cc:1428] Setting key f2bd30b601c16b80a15804622307e05e
I20260808 07:01:58.593626 19722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 440913078e1044f69dcc499818d4958b: Bootstrap starting.
I20260808 07:01:58.594451 19722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 440913078e1044f69dcc499818d4958b: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:58.594821 19722 log.cc:824] T 00000000000000000000000000000000 P 440913078e1044f69dcc499818d4958b: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:58.595805 19722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 440913078e1044f69dcc499818d4958b: No bootstrap required, opened a new log
I20260808 07:01:58.598215 19722 raft_consensus.cc:348] T 00000000000000000000000000000000 P 440913078e1044f69dcc499818d4958b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "440913078e1044f69dcc499818d4958b" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 34207 } }
I20260808 07:01:58.598374 19722 raft_consensus.cc:374] T 00000000000000000000000000000000 P 440913078e1044f69dcc499818d4958b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:58.598407 19722 raft_consensus.cc:729] T 00000000000000000000000000000000 P 440913078e1044f69dcc499818d4958b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 440913078e1044f69dcc499818d4958b, State: Initialized, Role: FOLLOWER
I20260808 07:01:58.598543 19722 consensus_queue.cc:260] T 00000000000000000000000000000000 P 440913078e1044f69dcc499818d4958b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "440913078e1044f69dcc499818d4958b" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 34207 } }
I20260808 07:01:58.598650 19722 raft_consensus.cc:388] T 00000000000000000000000000000000 P 440913078e1044f69dcc499818d4958b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:01:58.598723 19722 raft_consensus.cc:482] T 00000000000000000000000000000000 P 440913078e1044f69dcc499818d4958b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:01:58.598770 19722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 440913078e1044f69dcc499818d4958b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:58.599131 19725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:58.593005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44122 (local address 127.13.207.62:34207)
0808 07:01:58.593242 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:58.593250 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:58.593277 (+ 27us) server_negotiation.cc:407] Connection header received
0808 07:01:58.593691 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:58.593697 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:58.593794 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:58.594006 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:58.594275 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.595509 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.596607 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.597117 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.597222 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:58.598202 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:58.598206 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:58.598206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:58.598453 (+ 247us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:58.598526 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:58.598550 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:58.598622 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:58.598862 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 07:01:58.598963 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":110,"threads_started":1}
I20260808 07:01:58.599499 19722 raft_consensus.cc:504] T 00000000000000000000000000000000 P 440913078e1044f69dcc499818d4958b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "440913078e1044f69dcc499818d4958b" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 34207 } }
I20260808 07:01:58.599633 19722 leader_election.cc:302] T 00000000000000000000000000000000 P 440913078e1044f69dcc499818d4958b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 440913078e1044f69dcc499818d4958b; no voters:
I20260808 07:01:58.599864 19722 leader_election.cc:288] T 00000000000000000000000000000000 P 440913078e1044f69dcc499818d4958b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:01:58.599898 19727 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 440913078e1044f69dcc499818d4958b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:58.600033 19727 raft_consensus.cc:686] T 00000000000000000000000000000000 P 440913078e1044f69dcc499818d4958b [term 1 LEADER]: Becoming Leader. State: Replica: 440913078e1044f69dcc499818d4958b, State: Running, Role: LEADER
I20260808 07:01:58.600111 19727 consensus_queue.cc:237] T 00000000000000000000000000000000 P 440913078e1044f69dcc499818d4958b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "440913078e1044f69dcc499818d4958b" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 34207 } }
I20260808 07:01:58.600144 19722 sys_catalog.cc:564] T 00000000000000000000000000000000 P 440913078e1044f69dcc499818d4958b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:01:58.600322 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:01:58.600672 19729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 440913078e1044f69dcc499818d4958b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 440913078e1044f69dcc499818d4958b. Latest consensus state: current_term: 1 leader_uuid: "440913078e1044f69dcc499818d4958b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "440913078e1044f69dcc499818d4958b" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 34207 } } }
I20260808 07:01:58.600775 19729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 440913078e1044f69dcc499818d4958b [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:58.601018 19733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:01:58.601585 19733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:01:58.601888 19728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 440913078e1044f69dcc499818d4958b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "440913078e1044f69dcc499818d4958b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "440913078e1044f69dcc499818d4958b" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 34207 } } }
I20260808 07:01:58.601994 19728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 440913078e1044f69dcc499818d4958b [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:58.603598 19733 catalog_manager.cc:1347] Generated new cluster ID: c4fc441ed5fd41719ad57229a01d1943
I20260808 07:01:58.603681 19733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:01:58.619482 19733 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:01:58.620098 19733 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:01:58.627264 19733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 440913078e1044f69dcc499818d4958b: Generated new TSK 0
I20260808 07:01:58.627547 19733 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:01:58.644863 19725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:58.640356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44124 (local address 127.13.207.62:34207)
0808 07:01:58.640413 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:58.640415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:58.640520 (+ 105us) server_negotiation.cc:407] Connection header received
0808 07:01:58.640606 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:58.640607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:58.640617 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:58.640684 (+ 67us) server_negotiation.cc:226] Negotiated authn=JWT
0808 07:01:58.640845 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.641443 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.642619 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.642891 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.642953 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:58.643221 (+ 268us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 07:01:58.644605 (+ 1384us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43471/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 07:01:58.644764 (+ 159us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44124: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43471/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":30}
W20260808 07:01:58.644936 19752 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:58.640240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:34207 (local address 127.0.0.1:44124)
0808 07:01:58.640437 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:58.640450 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:58.640527 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:58.640707 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:58.640708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:58.640710 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 07:01:58.640777 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:58.640779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.641546 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.641547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:58.642531 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:58.642534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.642991 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.642992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:58.643106 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:58.643114 (+ 8us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 07:01:58.644716 (+ 1602us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43471/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 07:01:58.644818 (+ 102us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.207.62:34207: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43471/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":111,"threads_started":1}
W20260808 07:01:58.644953 19725 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44124: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43471/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260808 07:01:58.645538 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 19650
E20260808 07:01:58.656788 19753 webserver.cc:533] Webserver: sslize failed:
E20260808 07:01:58.656889 19753 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T07:01:58Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (251 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T07:01:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:01:58Z Disabled control of system clock
I20260808 07:01:58.695006 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:33855
--webserver_interface=127.13.207.62
--webserver_port=0
--builtin_ntp_servers=127.13.207.20:46553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:33855
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172493058068-14140-0/testchainca.cert with env {}
W20260808 07:01:58.784976 19757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:01:58.785457 19757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:01:58.785494 19757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:01:58.786965 19757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:01:58.787019 19757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:01:58.787032 19757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:01:58.787045 19757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:01:58.787058 19757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 07:01:58.787071 19757 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 07:01:58.787083 19757 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 07:01:58.789151 19757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:46553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:33855
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:33855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172493058068-14140-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:01:58.789489 19757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:01:58.789692 19757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:01:58.795576 19763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:58.795580 19762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:01:58.795580 19765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:01:58.795856 19757 server_base.cc:1034] running on GCE node
I20260808 07:01:58.796239 19757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:01:58.796693 19757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:01:58.797895 19757 hybrid_clock.cc:648] HybridClock initialized: now 1786172518797873 us; error 35 us; skew 500 ppm
I20260808 07:01:58.800194 19757 webserver.cc:460] Webserver started at http://127.13.207.62:36275/ using document root <none> and password file <none>
I20260808 07:01:58.801090 19757 fs_manager.cc:359] Metadata directory not provided
I20260808 07:01:58.801194 19757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:01:58.801369 19757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:01:58.802596 19757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "ddc95099977f4151bf6948e3c9311ff5"
format_stamp: "Formatted at 2026-08-08 07:01:58 on dist-test-slave-8tdl"
server_key: "47678628b4d4a49232a8a99e26e81811"
server_key_iv: "edb10e26a2358639e2f1554d2b9cb342"
server_key_version: "encryptionkey@0"
I20260808 07:01:58.803390 19757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "ddc95099977f4151bf6948e3c9311ff5"
format_stamp: "Formatted at 2026-08-08 07:01:58 on dist-test-slave-8tdl"
server_key: "47678628b4d4a49232a8a99e26e81811"
server_key_iv: "edb10e26a2358639e2f1554d2b9cb342"
server_key_version: "encryptionkey@0"
I20260808 07:01:58.805084 19757 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:01:58.806239 19771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:01:58.806664 19757 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:01:58.806874 19757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "ddc95099977f4151bf6948e3c9311ff5"
format_stamp: "Formatted at 2026-08-08 07:01:58 on dist-test-slave-8tdl"
server_key: "47678628b4d4a49232a8a99e26e81811"
server_key_iv: "edb10e26a2358639e2f1554d2b9cb342"
server_key_version: "encryptionkey@0"
I20260808 07:01:58.806973 19757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:01:58.823472 19757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:01:58.823890 19757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:01:58.824088 19757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:01:58.829674 19757 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:33855
I20260808 07:01:58.829723 19828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:33855 every 8 connection(s)
I20260808 07:01:58.830278 19757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:01:58.831094 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 19757
I20260808 07:01:58.831369 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:01:58.831733 14140 external_mini_cluster.cc:1428] Setting key 6d4dac029efe8eb8188283b40cc2323b
I20260808 07:01:58.832270 19829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:01:58.836537 19829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddc95099977f4151bf6948e3c9311ff5: Bootstrap starting.
I20260808 07:01:58.837436 19829 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddc95099977f4151bf6948e3c9311ff5: Neither blocks nor log segments found. Creating new log.
I20260808 07:01:58.837805 19829 log.cc:824] T 00000000000000000000000000000000 P ddc95099977f4151bf6948e3c9311ff5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:01:58.839134 19829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddc95099977f4151bf6948e3c9311ff5: No bootstrap required, opened a new log
I20260808 07:01:58.840718 19832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:58.832393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50584 (local address 127.13.207.62:33855)
0808 07:01:58.832755 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:58.832764 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:58.832802 (+ 38us) server_negotiation.cc:407] Connection header received
0808 07:01:58.833686 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:58.833696 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:58.833792 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:58.833964 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:01:58.834316 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.835857 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.837088 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.837761 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.837925 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:58.839634 (+ 1709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:01:58.839638 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:01:58.839639 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:01:58.839928 (+ 289us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:01:58.839973 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:01:58.840156 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:01:58.840290 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 07:01:58.840386 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 07:01:58.840492 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":186,"threads_started":1}
I20260808 07:01:58.841691 19829 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddc95099977f4151bf6948e3c9311ff5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddc95099977f4151bf6948e3c9311ff5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33855 } }
I20260808 07:01:58.841917 19829 raft_consensus.cc:374] T 00000000000000000000000000000000 P ddc95099977f4151bf6948e3c9311ff5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:01:58.841971 19829 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddc95099977f4151bf6948e3c9311ff5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddc95099977f4151bf6948e3c9311ff5, State: Initialized, Role: FOLLOWER
I20260808 07:01:58.842123 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:01:58.842113 19829 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddc95099977f4151bf6948e3c9311ff5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddc95099977f4151bf6948e3c9311ff5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33855 } }
I20260808 07:01:58.842216 19829 raft_consensus.cc:388] T 00000000000000000000000000000000 P ddc95099977f4151bf6948e3c9311ff5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:01:58.842264 19829 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddc95099977f4151bf6948e3c9311ff5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:01:58.842315 19829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddc95099977f4151bf6948e3c9311ff5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:01:58.843466 19829 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddc95099977f4151bf6948e3c9311ff5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddc95099977f4151bf6948e3c9311ff5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33855 } }
I20260808 07:01:58.843611 19829 leader_election.cc:302] T 00000000000000000000000000000000 P ddc95099977f4151bf6948e3c9311ff5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ddc95099977f4151bf6948e3c9311ff5; no voters:
I20260808 07:01:58.843827 19829 leader_election.cc:288] T 00000000000000000000000000000000 P ddc95099977f4151bf6948e3c9311ff5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:01:58.843902 19834 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ddc95099977f4151bf6948e3c9311ff5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:01:58.844156 19834 raft_consensus.cc:686] T 00000000000000000000000000000000 P ddc95099977f4151bf6948e3c9311ff5 [term 1 LEADER]: Becoming Leader. State: Replica: ddc95099977f4151bf6948e3c9311ff5, State: Running, Role: LEADER
I20260808 07:01:58.844161 19829 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddc95099977f4151bf6948e3c9311ff5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:01:58.844287 19834 consensus_queue.cc:237] T 00000000000000000000000000000000 P ddc95099977f4151bf6948e3c9311ff5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddc95099977f4151bf6948e3c9311ff5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33855 } }
I20260808 07:01:58.844780 19835 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddc95099977f4151bf6948e3c9311ff5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ddc95099977f4151bf6948e3c9311ff5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddc95099977f4151bf6948e3c9311ff5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33855 } } }
I20260808 07:01:58.844899 19835 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddc95099977f4151bf6948e3c9311ff5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:58.845237 19842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:01:58.845397 19836 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddc95099977f4151bf6948e3c9311ff5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddc95099977f4151bf6948e3c9311ff5. Latest consensus state: current_term: 1 leader_uuid: "ddc95099977f4151bf6948e3c9311ff5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddc95099977f4151bf6948e3c9311ff5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33855 } } }
I20260808 07:01:58.845528 19836 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddc95099977f4151bf6948e3c9311ff5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:01:58.845836 19842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:01:58.848393 19842 catalog_manager.cc:1347] Generated new cluster ID: 8195c9b10ca44404a07f01a335e49101
I20260808 07:01:58.848479 19842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:01:58.859639 19842 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:01:58.860584 19842 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:01:58.871305 19842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ddc95099977f4151bf6948e3c9311ff5: Generated new TSK 0
I20260808 07:01:58.872198 19842 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:01:59.687330 19832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:01:58.884482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50596 (local address 127.13.207.62:33855)
0808 07:01:58.884559 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:01:58.884561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:01:58.884666 (+ 105us) server_negotiation.cc:407] Connection header received
0808 07:01:58.884768 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:01:58.884769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:01:58.884786 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:01:58.884899 (+ 113us) server_negotiation.cc:226] Negotiated authn=JWT
0808 07:01:58.885349 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.886580 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.887769 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.888164 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.888253 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:58.888503 (+ 250us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 07:01:59.687039 (+798536us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 07:01:59.687258 (+ 219us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50596: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 07:01:59.687426 19832 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50596: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 07:01:59.687530 19859 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:01:58.884293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:33855 (local address 127.0.0.1:50596)
0808 07:01:58.884564 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 07:01:58.884577 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:01:58.884662 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:01:58.884922 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:01:58.884923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:01:58.884926 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0808 07:01:58.885003 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:58.885005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.886825 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.886827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:58.887574 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:01:58.887578 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:01:58.888296 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:01:58.888298 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:01:58.888358 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:01:58.888366 (+ 8us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 07:01:59.687240 (+798874us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 07:01:59.687381 (+ 141us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.207.62:33855: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":155,"threads_started":1}
I20260808 07:01:59.688211 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 19757
2026-08-08T07:01:59Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1022 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (108 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (105 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:01:59 dist-test-slave-8tdl krb5kdc[19871](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:01:59 dist-test-slave-8tdl krb5kdc[19871](info): set up 2 sockets
Aug 08 07:01:59 dist-test-slave-8tdl krb5kdc[19871](info): commencing operation
krb5kdc: starting...
W20260808 07:02:01.068238 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:02:01 dist-test-slave-8tdl krb5kdc[19871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:02:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:02:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:02:01.228041 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:39855
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:39855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:01.325294 19886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:01.325551 19886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:01.325577 19886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:01.326921 19886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:02:01.326983 19886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:01.326999 19886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:01.327013 19886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:02:01.327026 19886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:02:01.328750 19886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:39855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:39855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:01.329066 19886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:01.329376 19886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:01.335297 19891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:01.335302 19892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:01.335779 19886 server_base.cc:1034] running on GCE node
W20260808 07:02:01.335302 19894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:01.336109 19886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:01.336422 19886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:01.337605 19886 hybrid_clock.cc:648] HybridClock initialized: now 1786172521337556 us; error 57 us; skew 500 ppm
Aug 08 07:02:01 dist-test-slave-8tdl krb5kdc[19871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172521, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:01.340911 19886 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:02:01.341612 19886 webserver.cc:460] Webserver started at http://127.13.207.62:33411/ using document root <none> and password file <none>
I20260808 07:02:01.341900 19886 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:01.341984 19886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:01.342170 19886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:01.343520 19886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "bc5e8e47697e490fbeef52572ce11271"
format_stamp: "Formatted at 2026-08-08 07:02:01 on dist-test-slave-8tdl"
server_key: "edf94b68cf4791491790ca90deabaf83"
server_key_iv: "be5e05758ab59a9234a3e5b0a1260141"
server_key_version: "encryptionkey@0"
I20260808 07:02:01.343954 19886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "bc5e8e47697e490fbeef52572ce11271"
format_stamp: "Formatted at 2026-08-08 07:02:01 on dist-test-slave-8tdl"
server_key: "edf94b68cf4791491790ca90deabaf83"
server_key_iv: "be5e05758ab59a9234a3e5b0a1260141"
server_key_version: "encryptionkey@0"
I20260808 07:02:01.345845 19886 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 07:02:01.347234 19901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:01.347488 19886 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:01.347633 19886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "bc5e8e47697e490fbeef52572ce11271"
format_stamp: "Formatted at 2026-08-08 07:02:01 on dist-test-slave-8tdl"
server_key: "edf94b68cf4791491790ca90deabaf83"
server_key_iv: "be5e05758ab59a9234a3e5b0a1260141"
server_key_version: "encryptionkey@0"
I20260808 07:02:01.347747 19886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:01.371881 19886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:01.374581 19886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:01.374846 19886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:01.380194 19886 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:39855
I20260808 07:02:01.380254 19958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:39855 every 8 connection(s)
I20260808 07:02:01.380707 19886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:02:01.381845 19959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:01.384694 19959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc5e8e47697e490fbeef52572ce11271: Bootstrap starting.
I20260808 07:02:01.385521 19959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc5e8e47697e490fbeef52572ce11271: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:01.385646 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 19886
I20260808 07:02:01.385778 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:02:01.385959 19959 log.cc:824] T 00000000000000000000000000000000 P bc5e8e47697e490fbeef52572ce11271: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:01.386013 14140 external_mini_cluster.cc:1428] Setting key c7d36142e56dbb633dbae0baf48185a9
I20260808 07:02:01.387176 19959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc5e8e47697e490fbeef52572ce11271: No bootstrap required, opened a new log
I20260808 07:02:01.390769 19959 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc5e8e47697e490fbeef52572ce11271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc5e8e47697e490fbeef52572ce11271" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39855 } }
I20260808 07:02:01.391000 19959 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc5e8e47697e490fbeef52572ce11271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:01.391029 19959 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc5e8e47697e490fbeef52572ce11271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc5e8e47697e490fbeef52572ce11271, State: Initialized, Role: FOLLOWER
I20260808 07:02:01.391186 19959 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc5e8e47697e490fbeef52572ce11271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc5e8e47697e490fbeef52572ce11271" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39855 } }
I20260808 07:02:01.391270 19959 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc5e8e47697e490fbeef52572ce11271 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:02:01.391333 19959 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc5e8e47697e490fbeef52572ce11271 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:02:01.391403 19959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc5e8e47697e490fbeef52572ce11271 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:01.392125 19959 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc5e8e47697e490fbeef52572ce11271 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc5e8e47697e490fbeef52572ce11271" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39855 } }
I20260808 07:02:01.392257 19959 leader_election.cc:302] T 00000000000000000000000000000000 P bc5e8e47697e490fbeef52572ce11271 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc5e8e47697e490fbeef52572ce11271; no voters:
I20260808 07:02:01.392457 19959 leader_election.cc:288] T 00000000000000000000000000000000 P bc5e8e47697e490fbeef52572ce11271 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:02:01.392532 19964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc5e8e47697e490fbeef52572ce11271 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:02:01.392724 19964 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc5e8e47697e490fbeef52572ce11271 [term 1 LEADER]: Becoming Leader. State: Replica: bc5e8e47697e490fbeef52572ce11271, State: Running, Role: LEADER
I20260808 07:02:01.392802 19959 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc5e8e47697e490fbeef52572ce11271 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:02:01.392874 19964 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc5e8e47697e490fbeef52572ce11271 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc5e8e47697e490fbeef52572ce11271" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39855 } }
I20260808 07:02:01.393309 19964 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc5e8e47697e490fbeef52572ce11271 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc5e8e47697e490fbeef52572ce11271. Latest consensus state: current_term: 1 leader_uuid: "bc5e8e47697e490fbeef52572ce11271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc5e8e47697e490fbeef52572ce11271" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39855 } } }
I20260808 07:02:01.393399 19964 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc5e8e47697e490fbeef52572ce11271 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:01.393316 19965 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc5e8e47697e490fbeef52572ce11271 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc5e8e47697e490fbeef52572ce11271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc5e8e47697e490fbeef52572ce11271" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39855 } } }
I20260808 07:02:01.393728 19965 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc5e8e47697e490fbeef52572ce11271 [sys.catalog]: This master's current role is: LEADER
Aug 08 07:02:01 dist-test-slave-8tdl krb5kdc[19871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:02:01.394850 19968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:02:01.395737 19968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:02:01.397722 19968 catalog_manager.cc:1347] Generated new cluster ID: 6ed1e9129f884180bb1df3262fcdfada
I20260808 07:02:01.397794 19968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:02:01.399134 19963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:01.386657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46904 (local address 127.13.207.62:39855)
0808 07:02:01.386904 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:01.386918 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:01.386962 (+ 44us) server_negotiation.cc:407] Connection header received
0808 07:02:01.387627 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:01.387637 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:01.387745 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:01.387951 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:01.389014 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.390005 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.391026 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.391673 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.391806 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:01.394926 (+ 3120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:01.394932 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:01.394933 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:01.394947 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:01.397167 (+ 2220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:01.397683 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:01.397687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:01.397688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:01.397801 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:01.398073 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:01.398074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:01.398075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:01.398397 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:01.398554 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:01.398759 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 07:02:01.398907 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":106,"threads_started":1}
I20260808 07:02:01.405079 19968 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:02:01.405790 19968 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:02:01.409168 19968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc5e8e47697e490fbeef52572ce11271: Generated new TSK 0
I20260808 07:02:01.409405 19968 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
I20260808 07:02:01.454963 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:39855
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:01.554872 19985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:01.555124 19985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:01.555164 19985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:01.556511 19985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:01.556571 19985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:01.556602 19985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:02:01.558264 19985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:39855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:01.558539 19985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:01.558851 19985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:01.565225 19993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:01.565346 19991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:01.565405 19990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:01.565686 19985 server_base.cc:1034] running on GCE node
I20260808 07:02:01.566035 19985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:01.566359 19985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:01.567548 19985 hybrid_clock.cc:648] HybridClock initialized: now 1786172521567510 us; error 128 us; skew 500 ppm
Aug 08 07:02:01 dist-test-slave-8tdl krb5kdc[19871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172521, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:01.572468 19985 init.cc:377] Logged in from keytab as kudu/127.13.207.1@KRBTEST.COM (short username kudu)
I20260808 07:02:01.573508 19985 webserver.cc:460] Webserver started at http://127.13.207.1:35973/ using document root <none> and password file <none>
I20260808 07:02:01.573813 19985 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:01.573902 19985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:01.574150 19985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:01.575714 19985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "87ff89e92f274b208a6c16563424611f"
format_stamp: "Formatted at 2026-08-08 07:02:01 on dist-test-slave-8tdl"
server_key: "d39768cf8217eb592ee77c7549c9e601"
server_key_iv: "4231cc710350c7b71083c8b5b5cb7dff"
server_key_version: "encryptionkey@0"
I20260808 07:02:01.576205 19985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "87ff89e92f274b208a6c16563424611f"
format_stamp: "Formatted at 2026-08-08 07:02:01 on dist-test-slave-8tdl"
server_key: "d39768cf8217eb592ee77c7549c9e601"
server_key_iv: "4231cc710350c7b71083c8b5b5cb7dff"
server_key_version: "encryptionkey@0"
I20260808 07:02:01.578826 19985 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260808 07:02:01.580022 20000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:01.580297 19985 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:01.580452 19985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "87ff89e92f274b208a6c16563424611f"
format_stamp: "Formatted at 2026-08-08 07:02:01 on dist-test-slave-8tdl"
server_key: "d39768cf8217eb592ee77c7549c9e601"
server_key_iv: "4231cc710350c7b71083c8b5b5cb7dff"
server_key_version: "encryptionkey@0"
I20260808 07:02:01.580560 19985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:01.595624 19985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:01.598428 19985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:01.598620 19985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:01.598953 19985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:01.599300 19985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:01.599356 19985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:01.599395 19985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:01.599444 19985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:01.606890 19985 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:42701
I20260808 07:02:01.606980 20119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:42701 every 8 connection(s)
I20260808 07:02:01.607412 19985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:02:01.612890 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 19985
I20260808 07:02:01.613040 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:02:01.613325 14140 external_mini_cluster.cc:1428] Setting key f9bd42e5a83dc17304cd565f63e3cc2b
Aug 08 07:02:01 dist-test-slave-8tdl krb5kdc[19871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172521, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:02:01.620771 19963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:01.608583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:37113 (local address 127.13.207.62:39855)
0808 07:02:01.608636 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:01.608638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:01.609031 (+ 393us) server_negotiation.cc:407] Connection header received
0808 07:02:01.609963 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:01.609965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:01.609977 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:01.610047 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:01.611029 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.611667 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.613196 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.613783 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.613935 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:01.616898 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:01.616900 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:01.616901 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:01.616911 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:01.619131 (+ 2220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:01.619663 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:01.619665 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:01.619666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:01.619738 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:01.620038 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:01.620039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:01.620040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:01.620176 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:01.620259 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:01.620562 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 07:02:01.620669 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:02:01.620990 20122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:01.608502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:39855 (local address 127.13.207.1:37113)
0808 07:02:01.608695 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:01.608744 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:01.609788 (+ 1044us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:01.610113 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:01.610118 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:01.610577 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:01.610840 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:01.610847 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.611804 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.611806 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:01.612950 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:01.612954 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.614081 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.614082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:01.614241 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:01.614941 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:01.614952 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:01.616767 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:01.619249 (+ 2482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:01.619264 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:01.619267 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:01.619568 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:01.619821 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:01.619822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:01.619823 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:01.619955 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:01.620248 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:01.620256 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:01.620414 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 07:02:01.620536 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 07:02:01.620721 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":111,"threads_started":1}
I20260808 07:02:01.621690 20120 heartbeater.cc:347] Connected to a master server at 127.13.207.62:39855
I20260808 07:02:01.621826 20120 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:01.622062 20120 heartbeater.cc:511] Master 127.13.207.62:39855 requested a full tablet report, sending...
I20260808 07:02:01.622776 19920 ts_manager.cc:194] Registered new tserver with Master: 87ff89e92f274b208a6c16563424611f (127.13.207.1:42701)
I20260808 07:02:01.623461 19920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:37113
WARNING: no policy specified for kudu/127.13.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
I20260808 07:02:01.652068 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:39855
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:01.745873 20125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:01.746137 20125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:01.746176 20125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:01.747581 20125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:01.747643 20125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:01.747673 20125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:02:01.749374 20125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:39855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:01.749619 20125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:01.749828 20125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:01.755558 20130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:01.755556 20131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:01.755810 20125 server_base.cc:1034] running on GCE node
W20260808 07:02:01.755589 20133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:01.756126 20125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:01.756423 20125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:01.757642 20125 hybrid_clock.cc:648] HybridClock initialized: now 1786172521757605 us; error 65 us; skew 500 ppm
Aug 08 07:02:01 dist-test-slave-8tdl krb5kdc[19871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172521, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:01.761037 20125 init.cc:377] Logged in from keytab as kudu/127.13.207.2@KRBTEST.COM (short username kudu)
I20260808 07:02:01.761622 20125 webserver.cc:460] Webserver started at http://127.13.207.2:33427/ using document root <none> and password file <none>
I20260808 07:02:01.761893 20125 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:01.761948 20125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:01.762097 20125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:01.763445 20125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "b222c689c88142988d1e223a9c01dd90"
format_stamp: "Formatted at 2026-08-08 07:02:01 on dist-test-slave-8tdl"
server_key: "44a141923e158602efcaa011e710ddeb"
server_key_iv: "98b7083ff9331e34b383fdcb5a498b19"
server_key_version: "encryptionkey@0"
I20260808 07:02:01.763877 20125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "b222c689c88142988d1e223a9c01dd90"
format_stamp: "Formatted at 2026-08-08 07:02:01 on dist-test-slave-8tdl"
server_key: "44a141923e158602efcaa011e710ddeb"
server_key_iv: "98b7083ff9331e34b383fdcb5a498b19"
server_key_version: "encryptionkey@0"
I20260808 07:02:01.765663 20125 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 07:02:01.767357 20140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:01.767746 20125 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:01.768199 20125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "b222c689c88142988d1e223a9c01dd90"
format_stamp: "Formatted at 2026-08-08 07:02:01 on dist-test-slave-8tdl"
server_key: "44a141923e158602efcaa011e710ddeb"
server_key_iv: "98b7083ff9331e34b383fdcb5a498b19"
server_key_version: "encryptionkey@0"
I20260808 07:02:01.768317 20125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:01.784122 20125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:01.787431 20125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:01.787576 20125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:01.787786 20125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:01.788183 20125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:01.788250 20125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:01.788297 20125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:01.788343 20125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:01.798595 20125 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:39157
I20260808 07:02:01.798636 20255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:39157 every 8 connection(s)
I20260808 07:02:01.799080 20125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:02:01.799553 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 20125
I20260808 07:02:01.799679 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:02:01.799870 14140 external_mini_cluster.cc:1428] Setting key 6e8b6bb8143fac28c5e08a3bcd3af7c1
Aug 08 07:02:01 dist-test-slave-8tdl krb5kdc[19871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172521, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.207.3@KRBTEST.COM; defaulting to no policy
I20260808 07:02:01.814119 19963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:01.800754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:52549 (local address 127.13.207.62:39855)
0808 07:02:01.800820 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:01.800822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:01.801099 (+ 277us) server_negotiation.cc:407] Connection header received
0808 07:02:01.802337 (+ 1238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:01.802339 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:01.802362 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:01.802437 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:01.803299 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.804105 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.805978 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.806487 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.806616 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:01.810135 (+ 3519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:01.810137 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:01.810137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:01.810148 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:01.811927 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:01.812534 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:01.812536 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:01.812536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:01.812614 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:01.812929 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:01.812930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:01.812931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:01.813085 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:01.813226 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:01.813784 (+ 558us) server_negotiation.cc:299] Negotiation successful
0808 07:02:01.813990 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:02:01.814553 20258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:01.800748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:39855 (local address 127.13.207.2:52549)
0808 07:02:01.800943 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:01.800999 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:01.802195 (+ 1196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:01.802513 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:01.802517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:01.802959 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:01.803182 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:01.803187 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.804236 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.804238 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:01.805817 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:01.805822 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.806801 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.806802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:01.807005 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:01.807794 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:01.807809 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:01.809976 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:01.812095 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:01.812109 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:01.812112 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:01.812425 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:01.812697 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:01.812698 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:01.812698 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:01.812847 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:01.813169 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:01.813177 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:01.813405 (+ 228us) client_negotiation.cc:769] Sending connection context
0808 07:02:01.813752 (+ 347us) client_negotiation.cc:240] Negotiation successful
0808 07:02:01.814139 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":98,"threads_started":1}
I20260808 07:02:01.815515 20256 heartbeater.cc:347] Connected to a master server at 127.13.207.62:39855
I20260808 07:02:01.815765 20256 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.3@KRBTEST.COM" created.
I20260808 07:02:01.816143 20256 heartbeater.cc:511] Master 127.13.207.62:39855 requested a full tablet report, sending...
I20260808 07:02:01.816949 19920 ts_manager.cc:194] Registered new tserver with Master: b222c689c88142988d1e223a9c01dd90 (127.13.207.2:39157)
I20260808 07:02:01.817412 19920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:52549
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
I20260808 07:02:01.840293 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:39855
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:01.938203 20261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:01.938453 20261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:01.938482 20261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:01.939846 20261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:01.939917 20261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:01.939949 20261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:02:01.942075 20261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:39855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:01.942441 20261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:01.942801 20261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:01.949506 20267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:01.949471 20266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:01.949584 20269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:01.950604 20261 server_base.cc:1034] running on GCE node
I20260808 07:02:01.950959 20261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:01.951306 20261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:01.952579 20261 hybrid_clock.cc:648] HybridClock initialized: now 1786172521952501 us; error 98 us; skew 500 ppm
Aug 08 07:02:01 dist-test-slave-8tdl krb5kdc[19871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172521, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:01.955925 20261 init.cc:377] Logged in from keytab as kudu/127.13.207.3@KRBTEST.COM (short username kudu)
I20260808 07:02:01.956496 20261 webserver.cc:460] Webserver started at http://127.13.207.3:39903/ using document root <none> and password file <none>
I20260808 07:02:01.956775 20261 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:01.956837 20261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:01.957008 20261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:01.958319 20261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "eb22daab70d142beb8cc9b5e1abbdbc3"
format_stamp: "Formatted at 2026-08-08 07:02:01 on dist-test-slave-8tdl"
server_key: "26938b2d99ecfe8f159f44c51084c7a6"
server_key_iv: "2a373993197c9b3621063877466cc52d"
server_key_version: "encryptionkey@0"
I20260808 07:02:01.958817 20261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "eb22daab70d142beb8cc9b5e1abbdbc3"
format_stamp: "Formatted at 2026-08-08 07:02:01 on dist-test-slave-8tdl"
server_key: "26938b2d99ecfe8f159f44c51084c7a6"
server_key_iv: "2a373993197c9b3621063877466cc52d"
server_key_version: "encryptionkey@0"
I20260808 07:02:01.960569 20261 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 07:02:01.961633 20276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:01.962128 20261 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:01.962282 20261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "eb22daab70d142beb8cc9b5e1abbdbc3"
format_stamp: "Formatted at 2026-08-08 07:02:01 on dist-test-slave-8tdl"
server_key: "26938b2d99ecfe8f159f44c51084c7a6"
server_key_iv: "2a373993197c9b3621063877466cc52d"
server_key_version: "encryptionkey@0"
I20260808 07:02:01.962414 20261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:01.975986 20261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:01.979749 20261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:01.980156 20261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:01.980592 20261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:01.981377 20261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:01.981467 20261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:01.981509 20261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:01.981544 20261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:01.991065 20261 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:32935
I20260808 07:02:01.991145 20393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:32935 every 8 connection(s)
I20260808 07:02:01.991582 20261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:02:01.997998 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 20261
I20260808 07:02:01.998112 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:02:01.998306 14140 external_mini_cluster.cc:1428] Setting key 0cb9a107b3c6d4a53fb56eef3aaeed8c
Aug 08 07:02:01 dist-test-slave-8tdl krb5kdc[19871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172521, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:02:02.005805 19963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:01.992658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:53383 (local address 127.13.207.62:39855)
0808 07:02:01.992842 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:01.992845 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:01.993146 (+ 301us) server_negotiation.cc:407] Connection header received
0808 07:02:01.994552 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:01.994554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:01.994569 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:01.994929 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:01.995934 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.996522 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.998218 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.998534 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.998655 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:02.001822 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:02.001824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:02.001824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:02.001833 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:02.003824 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:02.004456 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:02.004457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:02.004458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:02.004524 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:02.004901 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:02.004902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:02.004903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:02.005059 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:02.005160 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:02.005550 (+ 390us) server_negotiation.cc:299] Negotiation successful
0808 07:02:02.005721 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260808 07:02:02.006137 20396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:01.992648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:39855 (local address 127.13.207.3:53383)
0808 07:02:01.992836 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:01.992884 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:01.994354 (+ 1470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:01.995078 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:01.995082 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:01.995540 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:01.995804 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:01.995811 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.996716 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.996718 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:01.998086 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:01.998091 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:01.998698 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:01.998700 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:01.998859 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:01.999591 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:01.999602 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:02.001647 (+ 2045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:02.003977 (+ 2330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:02.003988 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:02.003991 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:02.004304 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:02.004604 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:02.004605 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:02.004606 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:02.004740 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:02.005177 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:02.005185 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:02.005337 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 07:02:02.005460 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 07:02:02.005640 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260808 07:02:02.007086 20394 heartbeater.cc:347] Connected to a master server at 127.13.207.62:39855
I20260808 07:02:02.007211 20394 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:02.007439 20394 heartbeater.cc:511] Master 127.13.207.62:39855 requested a full tablet report, sending...
I20260808 07:02:02.008072 19920 ts_manager.cc:194] Registered new tserver with Master: eb22daab70d142beb8cc9b5e1abbdbc3 (127.13.207.3:32935)
I20260808 07:02:02.008517 19920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:53383
I20260808 07:02:02.010942 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/test-admin.keytab.
Aug 08 07:02:02 dist-test-slave-8tdl krb5kdc[19871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:02.033465 14140 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 07:02:02.035631 20401 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:02.034642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49406 (local address 127.13.207.1:42701)
0808 07:02:02.034891 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:02.034897 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:02.034908 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:02:02.034939 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:02.034942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:02.034961 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:02.035058 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:02.035522 (+ 464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49406: BlockingRecv error: recv got EOF from 127.0.0.1:49406 (error 108)
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":140,"threads_started":1}
W20260808 07:02:02.035771 14140 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/krb5cc' of type FILE
Aug 08 07:02:02 dist-test-slave-8tdl krb5kdc[19871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:02.037313 14140 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 07:02:02 dist-test-slave-8tdl krb5kdc[19871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:02.038661 14140 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 07:02:02 dist-test-slave-8tdl krb5kdc[19871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.1@KRBTEST.COM
I20260808 07:02:02.048796 20401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:02.039562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49416 (local address 127.13.207.1:42701)
0808 07:02:02.039635 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:02.039637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:02.039646 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:02:02.039679 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:02.039680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:02.039691 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:02.039755 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:02.040236 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:02.041023 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:02.042042 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:02.042415 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:02.042518 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:02.045345 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:02.045364 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:02.045364 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:02.045373 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:02.047314 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:02.047871 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:02.047875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:02.047875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:02.047940 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:02.048160 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:02.048161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:02.048161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:02.048335 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:02.048522 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:02.048569 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 07:02:02.048669 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 07:02:02.051532 20401 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:02.050613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49418 (local address 127.13.207.1:42701)
0808 07:02:02.050711 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:02.050713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:02.050723 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:02:02.050753 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:02.050754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:02.050764 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:02.050816 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:02.051440 (+ 624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49418: BlockingRecv error: recv got EOF from 127.0.0.1:49418 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 07:02:02.051604 14140 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/krb5cc' of type FILE
Aug 08 07:02:02 dist-test-slave-8tdl krb5kdc[19871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:02.053030 14140 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 07:02:02 dist-test-slave-8tdl krb5kdc[19871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:02.054111 14140 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 07:02:02 dist-test-slave-8tdl krb5kdc[19871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.1@KRBTEST.COM
I20260808 07:02:02.063573 20401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:02.055138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49434 (local address 127.13.207.1:42701)
0808 07:02:02.055212 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:02.055214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:02.055222 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:02:02.055251 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:02.055252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:02.055266 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:02.055335 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:02.056230 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:02.056762 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:02.057548 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:02.057774 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:02.057868 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:02.060337 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:02.060339 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:02.060339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:02.060350 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:02.062013 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:02.062554 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:02.062555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:02.062555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:02.062630 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:02.062894 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:02.062895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:02.062896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:02.063028 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:02.063166 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:02.063314 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 07:02:02.063441 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 07:02:02.066797 20401 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:02.065923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49440 (local address 127.13.207.1:42701)
0808 07:02:02.066098 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:02.066100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:02.066109 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:02:02.066134 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:02.066135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:02.066147 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:02.066246 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:02.066702 (+ 456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49440: BlockingRecv error: recv got EOF from 127.0.0.1:49440 (error 108)
Metrics: {"server-negotiator.queue_time_us":147}
W20260808 07:02:02.066999 14140 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172493058068-14140-0/krb5kdc/krb5cc' of type FILE
Aug 08 07:02:02 dist-test-slave-8tdl krb5kdc[19871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:02.068204 14140 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 07:02:02 dist-test-slave-8tdl krb5kdc[19871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:02.069221 14140 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 07:02:02 dist-test-slave-8tdl krb5kdc[19871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.1@KRBTEST.COM
I20260808 07:02:02.078064 20401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:02.069888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49444 (local address 127.13.207.1:42701)
0808 07:02:02.070025 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:02.070027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:02.070037 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:02:02.070061 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:02.070062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:02.070073 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:02.070153 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:02.070627 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:02.071252 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:02.072014 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:02.072235 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:02.072327 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:02.074944 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:02.074945 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:02.074946 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:02.074954 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:02.076705 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:02.077206 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:02.077207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:02.077207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:02.077277 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:02.077490 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:02.077491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:02.077491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:02.077615 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:02.077725 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:02.077857 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 07:02:02.077949 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260808 07:02:02.079324 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 19985
I20260808 07:02:02.086897 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 20125
I20260808 07:02:02.094519 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 20261
I20260808 07:02:02.103119 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 19886
2026-08-08T07:02:02Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2204 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:02:02 dist-test-slave-8tdl krb5kdc[20414](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:02:02 dist-test-slave-8tdl krb5kdc[20414](info): set up 2 sockets
Aug 08 07:02:02 dist-test-slave-8tdl krb5kdc[20414](info): commencing operation
krb5kdc: starting...
W20260808 07:02:03.275758 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:02:03 dist-test-slave-8tdl krb5kdc[20414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:02:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:02:03Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.62.keytab.
Entry for principal oryx/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.62.keytab.
I20260808 07:02:03.431466 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:35437
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.62.keytab
--principal=oryx/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:35437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:03.533656 20427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:03.534072 20427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:03.534128 20427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:03.537832 20427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:02:03.537947 20427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:03.537978 20427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:03.538018 20427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:02:03.538039 20427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:02:03.541067 20427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:35437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.62.keytab
--principal=oryx/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:35437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:03.541865 20427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:03.542431 20427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:03.548327 20433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:03.548311 20435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:03.548348 20432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:03.548697 20427 server_base.cc:1034] running on GCE node
I20260808 07:02:03.549002 20427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:03.549345 20427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:03.550552 20427 hybrid_clock.cc:648] HybridClock initialized: now 1786172523550530 us; error 42 us; skew 500 ppm
Aug 08 07:02:03 dist-test-slave-8tdl krb5kdc[20414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172523, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:03.553660 20427 init.cc:377] Logged in from keytab as oryx/127.13.207.62@KRBTEST.COM (short username oryx)
I20260808 07:02:03.554250 20427 webserver.cc:460] Webserver started at http://127.13.207.62:42873/ using document root <none> and password file <none>
I20260808 07:02:03.554507 20427 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:03.554589 20427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:03.554787 20427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:03.556011 20427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "109a60c497f74cb5bbb02a560ae83bc1"
format_stamp: "Formatted at 2026-08-08 07:02:03 on dist-test-slave-8tdl"
server_key: "887642a19bf8ac7c05af33c12b58f130"
server_key_iv: "430f6406366957c1788a716a344109b0"
server_key_version: "encryptionkey@0"
I20260808 07:02:03.556430 20427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "109a60c497f74cb5bbb02a560ae83bc1"
format_stamp: "Formatted at 2026-08-08 07:02:03 on dist-test-slave-8tdl"
server_key: "887642a19bf8ac7c05af33c12b58f130"
server_key_iv: "430f6406366957c1788a716a344109b0"
server_key_version: "encryptionkey@0"
I20260808 07:02:03.558646 20427 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 07:02:03.559798 20442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:03.560120 20427 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:03.560245 20427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "109a60c497f74cb5bbb02a560ae83bc1"
format_stamp: "Formatted at 2026-08-08 07:02:03 on dist-test-slave-8tdl"
server_key: "887642a19bf8ac7c05af33c12b58f130"
server_key_iv: "430f6406366957c1788a716a344109b0"
server_key_version: "encryptionkey@0"
I20260808 07:02:03.560338 20427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:03.584076 20427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:03.587097 20427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:03.587312 20427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:03.592105 20427 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:35437
I20260808 07:02:03.592147 20495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:35437 every 8 connection(s)
I20260808 07:02:03.592543 20427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:02:03.593624 20496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:03.596309 20496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1: Bootstrap starting.
I20260808 07:02:03.596975 20496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:03.597290 20496 log.cc:824] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:03.598138 20496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1: No bootstrap required, opened a new log
I20260808 07:02:03.598899 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 20427
I20260808 07:02:03.599026 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:02:03.599259 14140 external_mini_cluster.cc:1428] Setting key a25c688bb1d286562f8519eb0172db1a
I20260808 07:02:03.600811 20496 raft_consensus.cc:348] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } }
I20260808 07:02:03.600963 20496 raft_consensus.cc:374] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:03.600994 20496 raft_consensus.cc:729] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 109a60c497f74cb5bbb02a560ae83bc1, State: Initialized, Role: FOLLOWER
I20260808 07:02:03.601133 20496 consensus_queue.cc:260] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } }
I20260808 07:02:03.601224 20496 raft_consensus.cc:388] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:02:03.601271 20496 raft_consensus.cc:482] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:02:03.601323 20496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:03.602464 20496 raft_consensus.cc:504] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } }
I20260808 07:02:03.602588 20496 leader_election.cc:302] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 109a60c497f74cb5bbb02a560ae83bc1; no voters:
I20260808 07:02:03.602778 20496 leader_election.cc:288] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:02:03.602833 20501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:02:03.603034 20501 raft_consensus.cc:686] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [term 1 LEADER]: Becoming Leader. State: Replica: 109a60c497f74cb5bbb02a560ae83bc1, State: Running, Role: LEADER
I20260808 07:02:03.603197 20501 consensus_queue.cc:237] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } }
I20260808 07:02:03.603070 20496 sys_catalog.cc:564] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:02:03.604692 20502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "109a60c497f74cb5bbb02a560ae83bc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } }
I20260808 07:02:03.604873 20502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:03.604702 20503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 109a60c497f74cb5bbb02a560ae83bc1. Latest consensus state: current_term: 1 leader_uuid: "109a60c497f74cb5bbb02a560ae83bc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } }
I20260808 07:02:03.605202 20503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:03.605242 20517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:02:03.605897 20517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:02:03.607983 20517 catalog_manager.cc:1347] Generated new cluster ID: 8804463ead1f41d69e60b49669459c0f
I20260808 07:02:03.608068 20517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:02:03 dist-test-slave-8tdl krb5kdc[20414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM
I20260808 07:02:03.613163 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:03.600002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37560 (local address 127.13.207.62:35437)
0808 07:02:03.600273 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:03.600282 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:03.600310 (+ 28us) server_negotiation.cc:407] Connection header received
0808 07:02:03.600819 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:03.600827 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:03.600921 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:03.601138 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:03.602001 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:03.603442 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:03.604420 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:03.605081 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:03.605739 (+ 658us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:03.608794 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:03.608798 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:03.608799 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:03.608812 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:03.611101 (+ 2289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:03.611595 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:03.611598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:03.611599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:03.611755 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:03.612008 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:03.612008 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:03.612009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:03.612362 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:03.612540 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:03.612733 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 07:02:03.612904 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":159,"threads_started":1}
I20260808 07:02:03.618155 20517 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:02:03.618963 20517 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:02:03.629961 20517 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1: Generated new TSK 0
I20260808 07:02:03.630275 20517 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.1.keytab.
Entry for principal oryx/127.13.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.1.keytab.
I20260808 07:02:03.667695 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35437
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.1.keytab
--principal=oryx/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:03.761919 20523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:03.762168 20523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:03.762197 20523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:03.763661 20523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:03.763716 20523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:03.763747 20523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:02:03.765443 20523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.1.keytab
--principal=oryx/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:03.765760 20523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:03.766057 20523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:03.772662 20529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:03.772665 20531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:03.772666 20528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:03.773106 20523 server_base.cc:1034] running on GCE node
I20260808 07:02:03.773418 20523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:03.773715 20523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:03.774897 20523 hybrid_clock.cc:648] HybridClock initialized: now 1786172523774880 us; error 38 us; skew 500 ppm
Aug 08 07:02:03 dist-test-slave-8tdl krb5kdc[20414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172523, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:03.777792 20523 init.cc:377] Logged in from keytab as oryx/127.13.207.1@KRBTEST.COM (short username oryx)
I20260808 07:02:03.778366 20523 webserver.cc:460] Webserver started at http://127.13.207.1:45655/ using document root <none> and password file <none>
I20260808 07:02:03.778635 20523 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:03.778740 20523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:03.778899 20523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:03.780026 20523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "76046ba280e44aba95b3074f9a8701d1"
format_stamp: "Formatted at 2026-08-08 07:02:03 on dist-test-slave-8tdl"
server_key: "e2378cccf094eba97a0ec29df4f1ef9b"
server_key_iv: "179a95fb08361fc259ffc11f9f01bb87"
server_key_version: "encryptionkey@0"
I20260808 07:02:03.780417 20523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "76046ba280e44aba95b3074f9a8701d1"
format_stamp: "Formatted at 2026-08-08 07:02:03 on dist-test-slave-8tdl"
server_key: "e2378cccf094eba97a0ec29df4f1ef9b"
server_key_iv: "179a95fb08361fc259ffc11f9f01bb87"
server_key_version: "encryptionkey@0"
I20260808 07:02:03.782184 20523 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 07:02:03.783327 20538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:03.783658 20523 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:02:03.783782 20523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "76046ba280e44aba95b3074f9a8701d1"
format_stamp: "Formatted at 2026-08-08 07:02:03 on dist-test-slave-8tdl"
server_key: "e2378cccf094eba97a0ec29df4f1ef9b"
server_key_iv: "179a95fb08361fc259ffc11f9f01bb87"
server_key_version: "encryptionkey@0"
I20260808 07:02:03.783885 20523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:03.803017 20523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:03.805449 20523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:03.805675 20523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:03.805945 20523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:03.806300 20523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:03.806360 20523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:03.806397 20523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:03.806444 20523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:03.813475 20523 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:45561
I20260808 07:02:03.813524 20656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:45561 every 8 connection(s)
I20260808 07:02:03.813988 20523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:02:03.815219 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 20523
I20260808 07:02:03.815330 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:02:03.815582 14140 external_mini_cluster.cc:1428] Setting key c81da6e6dabec1835024e8b7dedbc5b1
Aug 08 07:02:03 dist-test-slave-8tdl krb5kdc[20414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172523, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM
WARNING: no policy specified for oryx/127.13.207.2@KRBTEST.COM; defaulting to no policy
I20260808 07:02:03.827456 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:03.815070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:35145 (local address 127.13.207.62:35437)
0808 07:02:03.815125 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:03.815127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:03.815491 (+ 364us) server_negotiation.cc:407] Connection header received
0808 07:02:03.816228 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:03.816229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:03.816240 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:03.816380 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:03.817445 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:03.818327 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:03.820288 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:03.820564 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:03.820634 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:03.823681 (+ 3047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:03.823684 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:03.823684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:03.823695 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:03.825638 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:03.826234 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:03.826236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:03.826236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:03.826306 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:03.826624 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:03.826625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:03.826626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:03.826838 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:03.826935 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:03.827245 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 07:02:03.827353 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:02:03.827684 20660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:03.814970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.13.207.1:35145)
0808 07:02:03.815318 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:03.815361 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:03.816090 (+ 729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:03.816476 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:03.816481 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:03.817120 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:03.817334 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:03.817338 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:03.818481 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:03.818483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:03.820143 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:03.820147 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:03.820686 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:03.820687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:03.820884 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:03.821534 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:03.821546 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:03.823550 (+ 2004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:03.825784 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:03.825794 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:03.825797 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:03.826133 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:03.826410 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:03.826411 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:03.826411 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:03.826528 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:03.826923 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:03.826931 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:03.827087 (+ 156us) client_negotiation.cc:769] Sending connection context
0808 07:02:03.827205 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 07:02:03.827400 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":117,"threads_started":1}
I20260808 07:02:03.828266 20657 heartbeater.cc:347] Connected to a master server at 127.13.207.62:35437
I20260808 07:02:03.828401 20657 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:03.828610 20657 heartbeater.cc:511] Master 127.13.207.62:35437 requested a full tablet report, sending...
I20260808 07:02:03.829273 20460 ts_manager.cc:194] Registered new tserver with Master: 76046ba280e44aba95b3074f9a8701d1 (127.13.207.1:45561)
I20260808 07:02:03.829947 20460 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:35145
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.2.keytab.
Entry for principal oryx/127.13.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.2.keytab.
I20260808 07:02:03.851199 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35437
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.2.keytab
--principal=oryx/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:03.947749 20663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:03.947988 20663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:03.948019 20663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:03.949251 20663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:03.949299 20663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:03.949329 20663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:02:03.951026 20663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.2.keytab
--principal=oryx/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:03.951261 20663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:03.951460 20663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:03.957440 20668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:03.957468 20669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:03.957444 20671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:03.958292 20663 server_base.cc:1034] running on GCE node
I20260808 07:02:03.958613 20663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:03.958984 20663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:03.960220 20663 hybrid_clock.cc:648] HybridClock initialized: now 1786172523960193 us; error 40 us; skew 500 ppm
Aug 08 07:02:03 dist-test-slave-8tdl krb5kdc[20414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172523, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:03.963207 20663 init.cc:377] Logged in from keytab as oryx/127.13.207.2@KRBTEST.COM (short username oryx)
I20260808 07:02:03.963928 20663 webserver.cc:460] Webserver started at http://127.13.207.2:44175/ using document root <none> and password file <none>
I20260808 07:02:03.964408 20663 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:03.964519 20663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:03.964694 20663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:03.965886 20663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "5426123c2e6b448d89af9dbe4b4c9a86"
format_stamp: "Formatted at 2026-08-08 07:02:03 on dist-test-slave-8tdl"
server_key: "3b61ac07d734998819c95dfc7300786f"
server_key_iv: "fd024b3d75e858a47613170887ce3698"
server_key_version: "encryptionkey@0"
I20260808 07:02:03.966306 20663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "5426123c2e6b448d89af9dbe4b4c9a86"
format_stamp: "Formatted at 2026-08-08 07:02:03 on dist-test-slave-8tdl"
server_key: "3b61ac07d734998819c95dfc7300786f"
server_key_iv: "fd024b3d75e858a47613170887ce3698"
server_key_version: "encryptionkey@0"
I20260808 07:02:03.968415 20663 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 07:02:03.969400 20678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:03.969587 20663 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:03.969710 20663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "5426123c2e6b448d89af9dbe4b4c9a86"
format_stamp: "Formatted at 2026-08-08 07:02:03 on dist-test-slave-8tdl"
server_key: "3b61ac07d734998819c95dfc7300786f"
server_key_iv: "fd024b3d75e858a47613170887ce3698"
server_key_version: "encryptionkey@0"
I20260808 07:02:03.969802 20663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:03.993637 20663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:03.996801 20663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:03.997051 20663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:03.997382 20663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:03.997761 20663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:03.997823 20663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:03.997867 20663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:03.997917 20663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:04.009478 20663 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:38057
I20260808 07:02:04.009598 20797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:38057 every 8 connection(s)
I20260808 07:02:04.009991 20663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:02:04 dist-test-slave-8tdl krb5kdc[20414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172523, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM
I20260808 07:02:04.018744 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 20663
I20260808 07:02:04.018879 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:02:04.019081 14140 external_mini_cluster.cc:1428] Setting key 114b862dfd1eb3a233e377d6592a5245
I20260808 07:02:04.023178 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.011238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:35183 (local address 127.13.207.62:35437)
0808 07:02:04.011519 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:04.011521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:04.011590 (+ 69us) server_negotiation.cc:407] Connection header received
0808 07:02:04.012237 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:04.012238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:04.012250 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:04.012317 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:04.013477 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.014188 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.015745 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.015954 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.016023 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.019268 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:04.019270 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:04.019270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:04.019281 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:04.020966 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.021617 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.021618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.021618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.021723 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.022063 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.022064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.022065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.022240 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:04.022496 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:04.022907 (+ 411us) server_negotiation.cc:299] Negotiation successful
0808 07:02:04.023054 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260808 07:02:04.023319 20800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.011204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.13.207.2:35183)
0808 07:02:04.011386 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:04.011431 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:04.012070 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:04.012442 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:04.012446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:04.013112 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:04.013341 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.013347 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.014451 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.014452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.015529 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.015533 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.016064 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.016065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.016195 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.016906 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:04.016917 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:04.019128 (+ 2211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:04.021085 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.021098 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.021101 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.021510 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.021820 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.021821 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.021822 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.021986 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.022512 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:04.022520 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:04.022673 (+ 153us) client_negotiation.cc:769] Sending connection context
0808 07:02:04.022861 (+ 188us) client_negotiation.cc:240] Negotiation successful
0808 07:02:04.023084 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":101,"threads_started":1}
I20260808 07:02:04.023877 20798 heartbeater.cc:347] Connected to a master server at 127.13.207.62:35437
I20260808 07:02:04.024024 20798 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:04.024233 20798 heartbeater.cc:511] Master 127.13.207.62:35437 requested a full tablet report, sending...
I20260808 07:02:04.024775 20460 ts_manager.cc:194] Registered new tserver with Master: 5426123c2e6b448d89af9dbe4b4c9a86 (127.13.207.2:38057)
I20260808 07:02:04.025339 20460 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:35183
WARNING: no policy specified for oryx/127.13.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.3.keytab.
Entry for principal oryx/127.13.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.3.keytab.
I20260808 07:02:04.056839 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35437
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.3.keytab
--principal=oryx/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:04.158612 20803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:04.158994 20803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:04.159027 20803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:04.160817 20803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:04.160912 20803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:04.160948 20803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:02:04.162829 20803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.3.keytab
--principal=oryx/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:04.163160 20803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:04.163399 20803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:04.171073 20808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:04.171073 20811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:04.171721 20809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:04.171985 20803 server_base.cc:1034] running on GCE node
I20260808 07:02:04.172271 20803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:04.172660 20803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:04.173862 20803 hybrid_clock.cc:648] HybridClock initialized: now 1786172524173850 us; error 37 us; skew 500 ppm
Aug 08 07:02:04 dist-test-slave-8tdl krb5kdc[20414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172524, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:04.176790 20803 init.cc:377] Logged in from keytab as oryx/127.13.207.3@KRBTEST.COM (short username oryx)
I20260808 07:02:04.177335 20803 webserver.cc:460] Webserver started at http://127.13.207.3:38005/ using document root <none> and password file <none>
I20260808 07:02:04.177596 20803 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:04.177678 20803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:04.177827 20803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:04.179065 20803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "acb5dce36f6043409bc52171d9ba692f"
format_stamp: "Formatted at 2026-08-08 07:02:04 on dist-test-slave-8tdl"
server_key: "a119febd3a084cefbe62674159a01cbc"
server_key_iv: "71c4942d0a68177a01fae605c9fd82c9"
server_key_version: "encryptionkey@0"
I20260808 07:02:04.179442 20803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "acb5dce36f6043409bc52171d9ba692f"
format_stamp: "Formatted at 2026-08-08 07:02:04 on dist-test-slave-8tdl"
server_key: "a119febd3a084cefbe62674159a01cbc"
server_key_iv: "71c4942d0a68177a01fae605c9fd82c9"
server_key_version: "encryptionkey@0"
I20260808 07:02:04.181084 20803 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:02:04.182257 20818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:04.182669 20803 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:04.183319 20803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "acb5dce36f6043409bc52171d9ba692f"
format_stamp: "Formatted at 2026-08-08 07:02:04 on dist-test-slave-8tdl"
server_key: "a119febd3a084cefbe62674159a01cbc"
server_key_iv: "71c4942d0a68177a01fae605c9fd82c9"
server_key_version: "encryptionkey@0"
I20260808 07:02:04.183490 20803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:04.205559 20803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:04.208484 20803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:04.208705 20803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:04.209000 20803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:04.209374 20803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:04.209504 20803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:04.209555 20803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:04.209583 20803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:04.216710 20803 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:46325
I20260808 07:02:04.216851 20936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:46325 every 8 connection(s)
I20260808 07:02:04.217180 20803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:02:04 dist-test-slave-8tdl krb5kdc[20414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172524, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM
I20260808 07:02:04.225054 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 20803
I20260808 07:02:04.225152 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:02:04.225561 14140 external_mini_cluster.cc:1428] Setting key 8b33d497102266c594484d6b738a3696
I20260808 07:02:04.229786 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.218143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:33619 (local address 127.13.207.62:35437)
0808 07:02:04.218241 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:04.218243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:04.218476 (+ 233us) server_negotiation.cc:407] Connection header received
0808 07:02:04.218981 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:04.218982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:04.218994 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:04.219074 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:04.220311 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.220814 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.221952 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.222189 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.222264 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.225680 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:04.225683 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:04.225683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:04.225695 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:04.227939 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.228466 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.228468 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.228468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.228536 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.229040 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.229041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.229042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.229182 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:04.229252 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:04.229626 (+ 374us) server_negotiation.cc:299] Negotiation successful
0808 07:02:04.229713 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 07:02:04.230007 20939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.218086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.13.207.3:33619)
0808 07:02:04.218260 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:04.218306 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:04.218862 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:04.219213 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:04.219218 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:04.219865 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:04.220086 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.220091 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.220953 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.220954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.221800 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.221803 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.222341 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.222343 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.222514 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.223281 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:04.223290 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:04.225501 (+ 2211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:04.228063 (+ 2562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.228073 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.228075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.228368 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.228752 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.228754 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.228754 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.228895 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.229281 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:04.229288 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:04.229475 (+ 187us) client_negotiation.cc:769] Sending connection context
0808 07:02:04.229577 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 07:02:04.229732 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
I20260808 07:02:04.230587 20937 heartbeater.cc:347] Connected to a master server at 127.13.207.62:35437
I20260808 07:02:04.230736 20937 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:04.230928 20937 heartbeater.cc:511] Master 127.13.207.62:35437 requested a full tablet report, sending...
I20260808 07:02:04.231386 20460 ts_manager.cc:194] Registered new tserver with Master: acb5dce36f6043409bc52171d9ba692f (127.13.207.3:46325)
I20260808 07:02:04.231832 20460 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:33619
I20260808 07:02:04.239326 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.207.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.207.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.207.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.61.keytab.
Entry for principal oryx/127.13.207.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.61.keytab.
I20260808 07:02:04.275137 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.61:38765
--webserver_interface=127.13.207.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.61.keytab
--principal=oryx/127.13.207.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:35437,127.13.207.61:38765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:04.370365 20942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:04.370633 20942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:04.370676 20942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:04.372094 20942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:02:04.372174 20942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:04.372200 20942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:04.372223 20942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:02:04.372263 20942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:02:04.373993 20942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:35437,127.13.207.61:38765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.61.keytab
--principal=oryx/127.13.207.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.61:38765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.207.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
W20260808 07:02:04.374155 20942 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.207.62:35437,127.13.207.61:38765'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 07:02:04.374327 20942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:04.374612 20942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:04.380182 20948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:04.380205 20951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:04.380208 20949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:04.380455 20942 server_base.cc:1034] running on GCE node
I20260808 07:02:04.380926 20942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:04.381278 20942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:04.382453 20942 hybrid_clock.cc:648] HybridClock initialized: now 1786172524382440 us; error 33 us; skew 500 ppm
Aug 08 07:02:04 dist-test-slave-8tdl krb5kdc[20414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172524, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:04.385701 20942 init.cc:377] Logged in from keytab as oryx/127.13.207.61@KRBTEST.COM (short username oryx)
I20260808 07:02:04.386251 20942 webserver.cc:460] Webserver started at http://127.13.207.61:35425/ using document root <none> and password file <none>
I20260808 07:02:04.386539 20942 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:04.386649 20942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:04.386828 20942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:04.388186 20942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/data/instance:
uuid: "92e4477c73b24415b051967647dac505"
format_stamp: "Formatted at 2026-08-08 07:02:04 on dist-test-slave-8tdl"
server_key: "bdf517be9bdb63a4d5c47d1815f5eeed"
server_key_iv: "67dac05bd2ae92de562154f120e71b60"
server_key_version: "encryptionkey@0"
I20260808 07:02:04.388794 20942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/wal/instance:
uuid: "92e4477c73b24415b051967647dac505"
format_stamp: "Formatted at 2026-08-08 07:02:04 on dist-test-slave-8tdl"
server_key: "bdf517be9bdb63a4d5c47d1815f5eeed"
server_key_iv: "67dac05bd2ae92de562154f120e71b60"
server_key_version: "encryptionkey@0"
I20260808 07:02:04.390739 20942 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 07:02:04.392016 20958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:04.392537 20942 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:04.392705 20942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/wal
uuid: "92e4477c73b24415b051967647dac505"
format_stamp: "Formatted at 2026-08-08 07:02:04 on dist-test-slave-8tdl"
server_key: "bdf517be9bdb63a4d5c47d1815f5eeed"
server_key_iv: "67dac05bd2ae92de562154f120e71b60"
server_key_version: "encryptionkey@0"
I20260808 07:02:04.392807 20942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:04.407392 20942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:04.410826 20942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:04.411041 20942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:04.416428 20942 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.61:38765
I20260808 07:02:04.416476 21010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.61:38765 every 8 connection(s)
I20260808 07:02:04.416968 20942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/data/info.pb
I20260808 07:02:04.418179 21011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:04.420420 21011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:02:04.422642 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 20942
Aug 08 07:02:04 dist-test-slave-8tdl krb5kdc[20414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172524, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.61@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM
Aug 08 07:02:04 dist-test-slave-8tdl krb5kdc[20414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.207.61@KRBTEST.COM
I20260808 07:02:04.433441 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.421485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37742 (local address 127.13.207.62:35437)
0808 07:02:04.421582 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:04.421585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:04.421866 (+ 281us) server_negotiation.cc:407] Connection header received
0808 07:02:04.422519 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:04.422520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:04.422537 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:04.422622 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:04.423713 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.424539 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.426195 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.426551 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.426644 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.429726 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:04.429728 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:04.429729 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:04.429739 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:04.431572 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.432105 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.432106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.432106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.432179 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.432544 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.432546 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.432546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.432803 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:04.432935 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:04.433286 (+ 351us) server_negotiation.cc:299] Negotiation successful
0808 07:02:04.433366 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 07:02:04.433810 21013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.421402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.0.0.1:37742)
0808 07:02:04.421698 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:04.421743 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:04.422330 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:04.422721 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:04.422725 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:04.423236 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:04.423538 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.423543 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.424666 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.424667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.426070 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.426072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.426667 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.426668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.426866 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.427575 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:04.427659 (+ 84us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:04.429603 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:04.431655 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.431664 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.431666 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.431978 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.432258 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.432259 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.432259 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.432437 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.432900 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:04.432908 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:04.433100 (+ 192us) client_negotiation.cc:769] Sending connection context
0808 07:02:04.433241 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 07:02:04.433499 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":184,"threads_started":1}
I20260808 07:02:04.434662 21011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:02:04.435590 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.423434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36408 (local address 127.13.207.61:38765)
0808 07:02:04.423665 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:04.423670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:04.423690 (+ 20us) server_negotiation.cc:407] Connection header received
0808 07:02:04.423712 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:04.423714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:04.423734 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:04.423808 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:04.424644 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.426009 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.427171 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.427848 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.427945 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.431453 (+ 3508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:04.431463 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:04.431463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:04.431471 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:04.433673 (+ 2202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.434293 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.434296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.434296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.434439 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.434669 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.434670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.434671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.434940 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:04.435082 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:04.435432 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 07:02:04.435502 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":139,"threads_started":1}
Aug 08 07:02:04 dist-test-slave-8tdl krb5kdc[20414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172524, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.61@KRBTEST.COM for oryx/127.13.207.61@KRBTEST.COM
I20260808 07:02:04.443914 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.435170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36418 (local address 127.13.207.61:38765)
0808 07:02:04.435352 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:04.435353 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:04.435358 (+ 5us) server_negotiation.cc:407] Connection header received
0808 07:02:04.435394 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:04.435395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:04.435404 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:04.435496 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:04.436113 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.436710 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.437574 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.437785 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.437861 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.440203 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:04.440204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:04.440205 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:04.440211 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:04.442421 (+ 2210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.442945 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.442946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.442946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.443068 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.443341 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.443341 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.443341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.443464 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:04.443531 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:04.443712 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 07:02:04.443810 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":167,"thread_start_us":129,"threads_started":1}
I20260808 07:02:04.444041 21013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.435115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:38765 (local address 127.0.0.1:36418)
0808 07:02:04.435162 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:04.435172 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:04.435211 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:04.435505 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:04.435506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:04.435815 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:04.435901 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.435903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.436807 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.436808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.437463 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.437465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.437891 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.437892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.437962 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.438547 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:04.438552 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:04.440109 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:04.442541 (+ 2432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.442542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.442543 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.442832 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.443196 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.443197 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.443198 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.443267 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.443586 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:04.443587 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:04.443630 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 07:02:04.443694 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 07:02:04.443899 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":112}
I20260808 07:02:04.445958 21011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505: Bootstrap starting.
I20260808 07:02:04.446875 21011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:04.447202 21011 log.cc:824] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:04.448328 21011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505: No bootstrap required, opened a new log
I20260808 07:02:04.450029 21011 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } }
I20260808 07:02:04.450189 21011 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:04.450215 21011 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92e4477c73b24415b051967647dac505, State: Initialized, Role: FOLLOWER
I20260808 07:02:04.450385 21011 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } }
I20260808 07:02:04.450865 21011 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:02:04.450867 21018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } } }
I20260808 07:02:04.451241 21018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:02:04.453316 21030 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:02:04.453395 21030 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:02:04.455826 20942 master_runner.cc:428] Detected that this master 92e4477c73b24415b051967647dac505 is joining an existing cluster
I20260808 07:02:04.455883 20942 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.207.61:38765
I20260808 07:02:04.464686 21036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.457054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.0.0.1:37746)
0808 07:02:04.457279 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:04.457293 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:04.457391 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:04.457617 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:04.457618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:04.457917 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:04.457989 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.457991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.458960 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.458961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.459565 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.459566 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.459978 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.459979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.460052 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.460673 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:04.460677 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:04.461252 (+ 575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:04.463259 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.463265 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.463266 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.463631 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.463886 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.463886 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.463887 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.463972 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.464271 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:04.464272 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:04.464319 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 07:02:04.464415 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 07:02:04.464552 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":35,"thread_start_us":132,"threads_started":1}
I20260808 07:02:04.464727 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.457129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37746 (local address 127.13.207.62:35437)
0808 07:02:04.457203 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:04.457206 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:04.457359 (+ 153us) server_negotiation.cc:407] Connection header received
0808 07:02:04.457493 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:04.457494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:04.457508 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:04.457543 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:04.458105 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.458866 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.459656 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.459872 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.459952 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.461368 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:04.461369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:04.461370 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:04.461378 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:04.463175 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.463727 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.463728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.463729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.463798 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.464033 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.464034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.464035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.464221 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:04.464286 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:04.464508 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 07:02:04.464610 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:02:04.464913 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.457369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36426 (local address 127.13.207.61:38765)
0808 07:02:04.457408 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:04.457410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:04.457552 (+ 142us) server_negotiation.cc:407] Connection header received
0808 07:02:04.457611 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:04.457612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:04.457622 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:04.457676 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:04.458047 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.458837 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.460103 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.460384 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.460463 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.461357 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:04.461358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:04.461359 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:04.461367 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:04.463618 (+ 2251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.464099 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.464107 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.464108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.464155 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.464390 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.464392 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.464392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.464510 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:04.464579 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:04.464747 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 07:02:04.464826 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:02:04.465199 21037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.457330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:38765 (local address 127.0.0.1:36426)
0808 07:02:04.457471 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:04.457478 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:04.457559 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:04.457681 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:04.457682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:04.457894 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:04.457962 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.457964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.458928 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.458929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.459959 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.459961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.460462 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.460463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.460528 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.460864 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:04.460868 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:04.461252 (+ 384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:04.463712 (+ 2460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.463714 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.463714 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.463972 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.464233 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.464233 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.464234 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.464325 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.464601 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:04.464601 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:04.464647 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 07:02:04.465001 (+ 354us) client_negotiation.cc:240] Negotiation successful
0808 07:02:04.465101 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":28,"thread_start_us":66,"threads_started":1}
Aug 08 07:02:04 dist-test-slave-8tdl krb5kdc[20414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172523, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.62@KRBTEST.COM for oryx/127.13.207.61@KRBTEST.COM
I20260808 07:02:04.475551 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.467600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36434 (local address 127.13.207.61:38765)
0808 07:02:04.467641 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:04.467643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:04.467843 (+ 200us) server_negotiation.cc:407] Connection header received
0808 07:02:04.467937 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:04.467938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:04.467949 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:04.467998 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:04.468469 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.468948 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.469760 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.470005 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.470086 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.472261 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:04.472262 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:04.472263 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:04.472271 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:04.474093 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.474568 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.474569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.474570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.474635 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.474934 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.474935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.474935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.475083 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:04.475126 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:04.475357 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 07:02:04.475448 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:02:04.475570 21038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.467575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:38765 (local address 127.0.0.1:36434)
0808 07:02:04.467722 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:04.467764 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:04.467870 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:04.468001 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:04.468004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:04.468311 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:04.468376 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.468378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.469036 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.469037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.469656 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.469659 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.470086 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.470087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.470178 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.470744 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:04.470753 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:04.472151 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:04.474198 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.474209 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.474212 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.474460 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.474746 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.474748 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.474748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.474831 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.475170 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:04.475177 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:04.475227 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 07:02:04.475316 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 07:02:04.475453 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
I20260808 07:02:04.476365 20460 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.207.61:38765: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "92e4477c73b24415b051967647dac505"
member_type: NON_VOTER
last_known_addr {
host: "127.13.207.61"
port: 38765
}
attrs {
promote: true
}
}
dest_uuid: "109a60c497f74cb5bbb02a560ae83bc1"
cas_config_opid_index: -1
I20260808 07:02:04.476708 20460 consensus_queue.cc:237] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: NON_VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: true } }
I20260808 07:02:04.477310 21040 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 92e4477c73b24415b051967647dac505 (127.13.207.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: NON_VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: true } } }
I20260808 07:02:04.477654 20982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:04.478281 21040 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.207.61:38765
I20260808 07:02:04.478379 21040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "109a60c497f74cb5bbb02a560ae83bc1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: NON_VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: true } } }
I20260808 07:02:04.478448 21040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:04.478629 20942 master.cc:537] Master@127.13.207.61:38765 shutting down...
I20260808 07:02:04.478767 20982 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [term 1 FOLLOWER]: Refusing update from remote peer 109a60c497f74cb5bbb02a560ae83bc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 07:02:04.479194 21039 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: NON_VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 07:02:04.479975 20445 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 07:02:04.480472 20445 consensus_peers.cc:487] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 -> Peer 92e4477c73b24415b051967647dac505 (127.13.207.61:38765): Couldn't send request to peer 92e4477c73b24415b051967647dac505. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 07:02:04.481711 20942 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:02:04.481824 20942 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:04.481882 20942 tablet_replica.cc:330] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505: stopping tablet replica
I20260808 07:02:04.495074 20942 master.cc:552] Master@127.13.207.61:38765 shutdown complete.
I20260808 07:02:04.495158 20942 master_runner.cc:305] Clearing existing system tablet
I20260808 07:02:04.495296 20942 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:04.495363 20942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:04.496024 20942 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:02:04.496568 21044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:04.496704 20942 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:04.496762 20942 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:02:04.496805 20942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/wal
uuid: "92e4477c73b24415b051967647dac505"
format_stamp: "Formatted at 2026-08-08 07:02:04 on dist-test-slave-8tdl"
server_key: "bdf517be9bdb63a4d5c47d1815f5eeed"
server_key_iv: "67dac05bd2ae92de562154f120e71b60"
server_key_version: "encryptionkey@0"
I20260808 07:02:04.497568 20942 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:04.499325 20942 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 07:02:04.499413 20942 log.cc:1196] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 07:02:04.499660 20942 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505: Deleting consensus metadata
I20260808 07:02:04.499799 20942 master_runner.cc:315] Copying system tablet from 127.13.207.62:35437
I20260808 07:02:04.500494 20942 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505: tablet copy: Beginning tablet copy session from remote peer at address 127.13.207.62:35437
I20260808 07:02:04.508824 21050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.500912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.0.0.1:37760)
0808 07:02:04.501111 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:04.501121 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:04.501193 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:04.501401 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:04.501401 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:04.501565 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:04.501868 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.501870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.503031 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.503033 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.503824 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.503829 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.504380 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.504381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.504452 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.505050 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:04.505055 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:04.505411 (+ 356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:04.507558 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.507559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.507560 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.507761 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.508042 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.508044 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.508044 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.508130 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.508489 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:04.508490 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:04.508562 (+ 72us) client_negotiation.cc:769] Sending connection context
0808 07:02:04.508626 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 07:02:04.508730 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":87,"threads_started":1}
I20260808 07:02:04.508906 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.500980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37760 (local address 127.13.207.62:35437)
0808 07:02:04.501044 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:04.501046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:04.501209 (+ 163us) server_negotiation.cc:407] Connection header received
0808 07:02:04.501253 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:04.501254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:04.501269 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:04.501397 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:04.501963 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.502910 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.503932 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.504281 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.504364 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.505521 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:04.505523 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:04.505523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:04.505533 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:04.507426 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.507846 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.507848 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.507848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.507923 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.508219 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.508220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.508220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.508408 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:04.508496 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:04.508682 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 07:02:04.508781 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:02:04.509604 20480 tablet_copy_service.cc:140] P 109a60c497f74cb5bbb02a560ae83bc1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 92e4477c73b24415b051967647dac505 ({username='oryx', principal='oryx/127.13.207.61@KRBTEST.COM'} at 127.0.0.1:37760)
I20260808 07:02:04.509730 20480 tablet_copy_service.cc:161] P 109a60c497f74cb5bbb02a560ae83bc1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 92e4477c73b24415b051967647dac505 at {username='oryx', principal='oryx/127.13.207.61@KRBTEST.COM'} at 127.0.0.1:37760: session id = 92e4477c73b24415b051967647dac505-00000000000000000000000000000000
I20260808 07:02:04.510762 20480 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 07:02:04.511587 20942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 07:02:04.512823 20942 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505: tablet copy: Starting download of 0 data blocks...
I20260808 07:02:04.513093 20942 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505: tablet copy: Starting download of 1 WAL segments...
I20260808 07:02:04.514380 20942 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 07:02:04.515535 20480 tablet_copy_service.cc:342] P 109a60c497f74cb5bbb02a560ae83bc1: Request end of tablet copy session 92e4477c73b24415b051967647dac505-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.207.61@KRBTEST.COM'} at 127.0.0.1:37760
I20260808 07:02:04.515617 20480 tablet_copy_service.cc:433] P 109a60c497f74cb5bbb02a560ae83bc1: ending tablet copy session 92e4477c73b24415b051967647dac505-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 92e4477c73b24415b051967647dac505
I20260808 07:02:04.519320 20942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:04.521133 21055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:04.521284 21051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:04.521733 21052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:04.521836 20942 server_base.cc:1034] running on GCE node
I20260808 07:02:04.521970 20942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:04.522153 20942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:04.523290 20942 hybrid_clock.cc:648] HybridClock initialized: now 1786172524523282 us; error 28 us; skew 500 ppm
Aug 08 07:02:04 dist-test-slave-8tdl krb5kdc[20414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172524, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:04.525225 20942 init.cc:377] Logged in from keytab as oryx/127.13.207.61@KRBTEST.COM (short username oryx)
I20260808 07:02:04.525676 20942 webserver.cc:460] Webserver started at http://127.13.207.61:34503/ using document root <none> and password file <none>
I20260808 07:02:04.525833 20942 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:04.525945 20942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:04.526759 20942 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:04.527544 21063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:04.527697 20942 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:02:04.527750 20942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/wal
uuid: "92e4477c73b24415b051967647dac505"
format_stamp: "Formatted at 2026-08-08 07:02:04 on dist-test-slave-8tdl"
server_key: "bdf517be9bdb63a4d5c47d1815f5eeed"
server_key_iv: "67dac05bd2ae92de562154f120e71b60"
server_key_version: "encryptionkey@0"
I20260808 07:02:04.527814 20942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:04.534032 20942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:04.536049 20942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:04.540419 20942 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.61:38765
I20260808 07:02:04.541090 21115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.61:38765 every 8 connection(s)
I20260808 07:02:04.541250 20942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-1/data/info.pb
I20260808 07:02:04.542407 21116 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:02:04.543022 21116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505: Bootstrap starting.
I20260808 07:02:04.546224 21116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:02:04.546497 21116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505: Bootstrap complete.
I20260808 07:02:04.546772 21116 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: NON_VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: true } }
I20260808 07:02:04.546871 21116 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92e4477c73b24415b051967647dac505, State: Initialized, Role: LEARNER
I20260808 07:02:04.546909 21116 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: NON_VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: true } }
I20260808 07:02:04.547180 21116 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:02:04.547137 21119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: NON_VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: true } } }
I20260808 07:02:04.547256 21119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [sys.catalog]: This master's current role is: LEARNER
I20260808 07:02:04.549064 21131 catalog_manager.cc:1259] Loaded cluster ID: 8804463ead1f41d69e60b49669459c0f
I20260808 07:02:04.549130 21131 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505: loading cluster ID for follower catalog manager: success
I20260808 07:02:04.550132 21131 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505: acquiring CA information for follower catalog manager: success
I20260808 07:02:04.550448 21131 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:02:04.596940 21132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.588223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36450 (local address 127.13.207.61:38765)
0808 07:02:04.588463 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:04.588466 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:04.588475 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:02:04.588511 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:04.588512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:04.588523 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:04.588614 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:04.589427 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.590194 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.591030 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.591253 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.591370 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.593112 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:04.593114 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:04.593115 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:04.593124 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:04.595214 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.595675 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.595677 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.595678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.595759 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.596069 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.596070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.596070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.596239 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:04.596360 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:04.596552 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 07:02:04.596656 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":101,"threads_started":1}
WARNING: no policy specified for oryx/127.13.207.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.207.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.207.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.60.keytab.
Entry for principal oryx/127.13.207.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.60.keytab.
I20260808 07:02:04.630358 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.60:32895
--webserver_interface=127.13.207.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.60.keytab
--principal=oryx/127.13.207.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:35437,127.13.207.61:38765,127.13.207.60:32895
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:04.719039 21135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:04.719290 21135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:04.719316 21135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:04.720772 21135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:02:04.720846 21135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:04.720862 21135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:04.720876 21135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:02:04.720888 21135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:02:04.722718 21135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:35437,127.13.207.61:38765,127.13.207.60:32895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.60.keytab
--principal=oryx/127.13.207.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.60:32895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.207.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:04.723029 21135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:04.723299 21135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:04.728518 21140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:04.728528 21141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:04.728536 21143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:04.729015 21135 server_base.cc:1034] running on GCE node
I20260808 07:02:04.729283 21135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:04.729599 21135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:04.730775 21135 hybrid_clock.cc:648] HybridClock initialized: now 1786172524730743 us; error 44 us; skew 500 ppm
Aug 08 07:02:04 dist-test-slave-8tdl krb5kdc[20414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172524, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:04.733698 21135 init.cc:377] Logged in from keytab as oryx/127.13.207.60@KRBTEST.COM (short username oryx)
I20260808 07:02:04.734434 21135 webserver.cc:460] Webserver started at http://127.13.207.60:34357/ using document root <none> and password file <none>
I20260808 07:02:04.734777 21135 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:04.734871 21135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:04.735025 21135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:04.736194 21135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/data/instance:
uuid: "e93d3d4db1114d719e7c2d6f99456785"
format_stamp: "Formatted at 2026-08-08 07:02:04 on dist-test-slave-8tdl"
server_key: "7b4a97cf0114615a2d33cdaec37af1bf"
server_key_iv: "fd7eeda3c79a5b9cb74e3aefcf1364e8"
server_key_version: "encryptionkey@0"
I20260808 07:02:04.736639 21135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/wal/instance:
uuid: "e93d3d4db1114d719e7c2d6f99456785"
format_stamp: "Formatted at 2026-08-08 07:02:04 on dist-test-slave-8tdl"
server_key: "7b4a97cf0114615a2d33cdaec37af1bf"
server_key_iv: "fd7eeda3c79a5b9cb74e3aefcf1364e8"
server_key_version: "encryptionkey@0"
I20260808 07:02:04.738430 21135 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 07:02:04.739599 21150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:04.739884 21135 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:02:04.740028 21135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/wal
uuid: "e93d3d4db1114d719e7c2d6f99456785"
format_stamp: "Formatted at 2026-08-08 07:02:04 on dist-test-slave-8tdl"
server_key: "7b4a97cf0114615a2d33cdaec37af1bf"
server_key_iv: "fd7eeda3c79a5b9cb74e3aefcf1364e8"
server_key_version: "encryptionkey@0"
I20260808 07:02:04.740134 21135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:04.780382 21135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:04.783808 21135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:04.784039 21135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:04.790473 21135 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.60:32895
I20260808 07:02:04.790632 21204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.60:32895 every 8 connection(s)
I20260808 07:02:04.791007 21135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/data/info.pb
I20260808 07:02:04.792399 21205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:04.794132 21205 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:02:04.797480 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 21135
Aug 08 07:02:04 dist-test-slave-8tdl krb5kdc[20414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172524, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.60@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM
Aug 08 07:02:04 dist-test-slave-8tdl krb5kdc[20414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.207.60@KRBTEST.COM
I20260808 07:02:04.807030 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.795204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37764 (local address 127.13.207.62:35437)
0808 07:02:04.795275 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:04.795277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:04.795479 (+ 202us) server_negotiation.cc:407] Connection header received
0808 07:02:04.796006 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:04.796008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:04.796019 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:04.796090 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:04.796807 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.797659 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.799697 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.800021 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.800108 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.802762 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:04.802764 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:04.802764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:04.802775 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:04.804681 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.805371 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.805372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.805373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.805459 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.805943 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.805945 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.805945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.806212 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:04.806294 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:04.806835 (+ 541us) server_negotiation.cc:299] Negotiation successful
0808 07:02:04.806933 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:02:04.807230 21207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.795102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.0.0.1:37764)
0808 07:02:04.795324 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:04.795371 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:04.795866 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:04.796211 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:04.796215 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:04.796484 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:04.796708 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.796712 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.797778 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.797780 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.799582 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.799586 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.800147 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.800148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.800260 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.800930 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:04.800940 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:04.802613 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:04.804807 (+ 2194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.804815 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.804818 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.805242 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.805628 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.805629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.805630 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.805833 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.806310 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:04.806318 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:04.806569 (+ 251us) client_negotiation.cc:769] Sending connection context
0808 07:02:04.806747 (+ 178us) client_negotiation.cc:240] Negotiation successful
0808 07:02:04.806967 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":135,"threads_started":1}
I20260808 07:02:04.808079 21205 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:02:04.808784 21208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.798258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36788 (local address 127.13.207.60:32895)
0808 07:02:04.798509 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:04.798514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:04.798788 (+ 274us) server_negotiation.cc:407] Connection header received
0808 07:02:04.798822 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:04.798824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:04.798847 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:04.798919 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:04.799752 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.800650 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.801414 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.801730 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.801817 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.805038 (+ 3221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:04.805045 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:04.805045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:04.805053 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:04.807281 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.807743 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.807746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.807747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.807809 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.808046 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.808048 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.808048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.808256 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:04.808385 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:04.808545 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 07:02:04.808647 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":145,"threads_started":1}
Aug 08 07:02:04 dist-test-slave-8tdl krb5kdc[20414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172524, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.60@KRBTEST.COM for oryx/127.13.207.61@KRBTEST.COM
I20260808 07:02:04.817082 21207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.808428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:38765 (local address 127.0.0.1:36456)
0808 07:02:04.808479 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:04.808489 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:04.808523 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:04.808772 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:04.808773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:04.808997 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:04.809036 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.809037 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.809865 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.809866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.810423 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.810426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.811039 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.811041 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.811102 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.811590 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:04.811596 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:04.813051 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:04.815576 (+ 2525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.815577 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.815577 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.815804 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.816108 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.816109 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.816110 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.816198 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.816770 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:04.816772 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:04.816822 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 07:02:04.816889 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 07:02:04.817000 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 07:02:04.817226 21132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.808567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36456 (local address 127.13.207.61:38765)
0808 07:02:04.808614 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:04.808616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:04.808624 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:02:04.808651 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:04.808652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:04.808664 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:04.808725 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:04.809137 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.809745 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.810596 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.810931 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.811009 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.813208 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:04.813209 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:04.813210 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:04.813219 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:04.815423 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.815924 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.815925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.815926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.815994 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.816308 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.816310 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.816310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.816514 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:04.816595 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:04.816978 (+ 383us) server_negotiation.cc:299] Negotiation successful
0808 07:02:04.817124 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:02:04.817966 21205 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.60" port: 32895 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:02:04 dist-test-slave-8tdl krb5kdc[20414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172524, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.60@KRBTEST.COM for oryx/127.13.207.60@KRBTEST.COM
I20260808 07:02:04.826159 21207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.818360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:32895 (local address 127.0.0.1:36794)
0808 07:02:04.818439 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:04.818452 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:04.818509 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:04.818760 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:04.818761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:04.818908 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:04.818945 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.818946 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.819827 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.819828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.820355 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.820357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.820784 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.820785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.820836 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.821413 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:04.821418 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:04.822810 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:04.824783 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.824784 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.824784 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.824990 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.825388 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.825389 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.825389 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.825458 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.825797 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:04.825798 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:04.825889 (+ 91us) client_negotiation.cc:769] Sending connection context
0808 07:02:04.825976 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 07:02:04.826073 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 07:02:04.826277 21208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.818531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36794 (local address 127.13.207.60:32895)
0808 07:02:04.818612 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:04.818614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:04.818620 (+ 6us) server_negotiation.cc:407] Connection header received
0808 07:02:04.818636 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:04.818637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:04.818646 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:04.818755 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:04.819134 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.819705 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.820450 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.820688 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.820787 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.823056 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:04.823058 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:04.823058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:04.823065 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:04.824675 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.825112 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.825114 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.825114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.825249 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.825558 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.825559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.825559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.825702 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:04.825766 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:04.826049 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 07:02:04.826134 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 07:02:04.828302 21205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785: Bootstrap starting.
I20260808 07:02:04.829200 21205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:04.829516 21205 log.cc:824] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:04.830804 21205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785: No bootstrap required, opened a new log
I20260808 07:02:04.831104 20657 heartbeater.cc:503] Master 127.13.207.62:35437 was elected leader, sending a full tablet report...
I20260808 07:02:04.832736 21205 raft_consensus.cc:348] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 32895 } }
I20260808 07:02:04.832923 21205 raft_consensus.cc:374] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:04.832978 21205 raft_consensus.cc:729] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e93d3d4db1114d719e7c2d6f99456785, State: Initialized, Role: FOLLOWER
I20260808 07:02:04.833150 21205 consensus_queue.cc:260] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 32895 } }
I20260808 07:02:04.833676 21205 sys_catalog.cc:564] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:02:04.833695 21210 sys_catalog.cc:455] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 32895 } } }
I20260808 07:02:04.833858 21210 sys_catalog.cc:458] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:02:04.836215 21221 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:02:04.836298 21221 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:02:04.839468 21135 master_runner.cc:428] Detected that this master e93d3d4db1114d719e7c2d6f99456785 is joining an existing cluster
I20260808 07:02:04.839552 21135 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.207.60:32895
I20260808 07:02:04.848982 21208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.841375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36798 (local address 127.13.207.60:32895)
0808 07:02:04.841418 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:04.841420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:04.841640 (+ 220us) server_negotiation.cc:407] Connection header received
0808 07:02:04.841739 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:04.841740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:04.841748 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:04.841778 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:04.842211 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.843000 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.844123 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.844452 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.844500 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.845832 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:04.845834 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:04.845834 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:04.845842 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:04.847614 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.847989 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.847991 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.847991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.848060 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.848329 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.848329 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.848330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.848476 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:04.848540 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:04.848729 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 07:02:04.848886 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":73,"server-negotiator.queue_time_us":22}
I20260808 07:02:04.848985 21230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.841456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:32895 (local address 127.0.0.1:36798)
0808 07:02:04.841578 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:04.841588 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:04.841664 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:04.841832 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:04.841833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:04.842065 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:04.842120 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.842122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.843084 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.843084 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.844038 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.844039 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.844682 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.844683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.844748 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.845311 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:04.845316 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:04.845717 (+ 401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:04.847716 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.847717 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.847717 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.847927 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.848174 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.848175 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.848175 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.848243 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.848550 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:04.848551 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:04.848614 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 07:02:04.848696 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:02:04.848821 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":69,"threads_started":1}
I20260808 07:02:04.860988 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 20523
I20260808 07:02:04.871727 21132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.841098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36472 (local address 127.13.207.61:38765)
0808 07:02:04.841154 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:04.841156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:04.841325 (+ 169us) server_negotiation.cc:407] Connection header received
0808 07:02:04.841448 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:04.841449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:04.841460 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:04.841492 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:04.841993 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.842807 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.844130 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.844457 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.844502 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.867312 (+ 22810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:04.867315 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:04.867315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:04.867324 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:04.870279 (+ 2955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.870762 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.870764 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.870765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.870829 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.871134 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.871135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.871135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.871277 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:04.871319 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:04.871532 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 07:02:04.871625 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:02:04.871754 21228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.840735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.0.0.1:37778)
0808 07:02:04.840985 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:04.840998 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:04.841147 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:04.841378 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:04.841379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:04.841707 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:04.841810 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.841812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.842972 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.842973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.844003 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.844006 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.844603 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.844604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.844674 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.865478 (+ 20804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:04.865486 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:04.866012 (+ 526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:04.869890 (+ 3878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.869892 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.869893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.870206 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.870668 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.870670 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.870670 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.870782 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.871169 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:04.871170 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:04.871231 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 07:02:04.871306 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 07:02:04.871458 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":3,"thread_start_us":129,"threads_started":1}
I20260808 07:02:04.872253 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.840833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37778 (local address 127.13.207.62:35437)
0808 07:02:04.840910 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:04.840912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:04.841180 (+ 268us) server_negotiation.cc:407] Connection header received
0808 07:02:04.841216 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:04.841218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:04.841232 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:04.841302 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:04.841911 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.842878 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.844114 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.844462 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.844526 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.866228 (+ 21702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:04.866230 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:04.866231 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:04.866243 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:04.869646 (+ 3403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.870317 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.870319 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.870319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.870396 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.870872 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.870874 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.870874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.871060 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:04.871137 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:04.871917 (+ 780us) server_negotiation.cc:299] Negotiation successful
0808 07:02:04.872111 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:02:04.872779 21229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.841032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:38765 (local address 127.0.0.1:36472)
0808 07:02:04.841258 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:04.841267 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:04.841364 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:04.841564 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:04.841565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:04.841819 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:04.841891 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.841893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.842914 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.842915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.844001 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.844004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.844671 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.844672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.844742 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.866730 (+ 21988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:04.866736 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:04.867165 (+ 429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:04.870394 (+ 3229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.870395 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.870395 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.870620 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.870993 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.870994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.870994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.871059 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.871366 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:04.871367 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:04.871413 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 07:02:04.872268 (+ 855us) client_negotiation.cc:240] Negotiation successful
0808 07:02:04.872665 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":46,"thread_start_us":182,"threads_started":1}
I20260808 07:02:04.872797 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:45561
--local_ip_for_outbound_sockets=127.13.207.1
--tserver_master_addrs=127.13.207.62:35437
--webserver_port=45655
--webserver_interface=127.13.207.1
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.1.keytab
--principal=oryx/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.207.62:35437,127.13.207.61:38765
--tserver_master_addrs=127.13.207.62:35437,127.13.207.61:38765,127.13.207.60:32895 with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:02:04 dist-test-slave-8tdl krb5kdc[20414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172523, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.62@KRBTEST.COM for oryx/127.13.207.60@KRBTEST.COM
I20260808 07:02:04.884935 21038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.875053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:32895 (local address 127.0.0.1:36804)
0808 07:02:04.875137 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:04.875152 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:04.875209 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:04.875387 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:04.875388 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:04.875621 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:04.875685 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.875688 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.876619 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.876621 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.877684 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.877689 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.878220 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.878222 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.878327 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.879114 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:04.879122 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:04.881002 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:04.883519 (+ 2517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.883521 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.883521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.883796 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.884068 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.884069 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.884069 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.884149 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.884538 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:04.884539 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:04.884598 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 07:02:04.884678 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:02:04.884817 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 07:02:04.884954 21208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.875162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36804 (local address 127.13.207.60:32895)
0808 07:02:04.875202 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:04.875204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:04.875213 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:02:04.875294 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:04.875295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:04.875305 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:04.875365 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:04.875775 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.876521 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.877800 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.878111 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.878194 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.881136 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:04.881138 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:04.881138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:04.881149 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:04.883345 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.883896 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.883899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.883900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.883978 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.884231 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.884232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.884232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.884429 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:04.884518 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:04.884725 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 07:02:04.884823 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:02:04.886297 20460 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.207.60:32895: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785"
member_type: NON_VOTER
last_known_addr {
host: "127.13.207.60"
port: 32895
}
attrs {
promote: true
}
}
dest_uuid: "109a60c497f74cb5bbb02a560ae83bc1"
cas_config_opid_index: 5
I20260808 07:02:04.886601 20460 consensus_queue.cc:237] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: NON_VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: true } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: NON_VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: true } }
I20260808 07:02:04.887156 21040 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e93d3d4db1114d719e7c2d6f99456785 (127.13.207.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: NON_VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: true } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: NON_VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: true } } }
I20260808 07:02:04.887692 21178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:04.888207 21040 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.207.60:32895
I20260808 07:02:04.888305 21040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "109a60c497f74cb5bbb02a560ae83bc1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: NON_VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: true } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: NON_VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: true } } }
I20260808 07:02:04.888374 21040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:04.889343 21135 master.cc:537] Master@127.13.207.60:32895 shutting down...
I20260808 07:02:04.889751 21178 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [term 1 FOLLOWER]: Refusing update from remote peer 109a60c497f74cb5bbb02a560ae83bc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 07:02:04.890158 21039 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: NON_VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 07:02:04.891448 20445 consensus_peers.cc:487] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 -> Peer e93d3d4db1114d719e7c2d6f99456785 (127.13.207.60:32895): Couldn't send request to peer e93d3d4db1114d719e7c2d6f99456785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 07:02:04.901605 21135 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:02:04.901747 21135 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:04.901800 21135 tablet_replica.cc:330] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785: stopping tablet replica
I20260808 07:02:04.902154 21132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.887352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36480 (local address 127.13.207.61:38765)
0808 07:02:04.889105 (+ 1753us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:04.889107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:04.891627 (+ 2520us) server_negotiation.cc:407] Connection header received
0808 07:02:04.891661 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:04.891662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:04.891691 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:04.891748 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:04.892526 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.893315 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.900425 (+ 7110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.901099 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.901173 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.901275 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:04.902016 (+ 741us) server_negotiation.cc:299] Negotiation successful
0808 07:02:04.902077 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1539}
I20260808 07:02:04.902158 21038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.887224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:38765 (local address 127.0.0.1:36480)
0808 07:02:04.891495 (+ 4271us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:04.891510 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:04.891566 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:04.892212 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:04.892213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:04.892215 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:04.892271 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.892273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.893441 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.893443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.900238 (+ 6795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.900247 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.901605 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.901606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.901875 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.901927 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 07:02:04.901999 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 07:02:04.902066 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4248}
I20260808 07:02:04.902788 21090 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [term 1 LEARNER]: Refusing update from remote peer 109a60c497f74cb5bbb02a560ae83bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 07:02:04.903045 21039 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: NON_VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 07:02:04.903743 21090 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e93d3d4db1114d719e7c2d6f99456785 (127.13.207.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: NON_VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: true } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: NON_VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: true } } }
I20260808 07:02:04.906301 21119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 109a60c497f74cb5bbb02a560ae83bc1. Latest consensus state: current_term: 1 leader_uuid: "109a60c497f74cb5bbb02a560ae83bc1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: NON_VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: true } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: NON_VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: true } } }
I20260808 07:02:04.906612 21119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [sys.catalog]: This master's current role is: LEARNER
I20260808 07:02:04.906713 21119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "109a60c497f74cb5bbb02a560ae83bc1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: NON_VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: true } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: NON_VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: true } } }
I20260808 07:02:04.906769 21119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [sys.catalog]: This master's current role is: LEARNER
I20260808 07:02:04.907140 21039 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1: attempting to promote NON_VOTER 92e4477c73b24415b051967647dac505 to VOTER
I20260808 07:02:04.907660 21039 consensus_queue.cc:237] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: NON_VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: true } }
W20260808 07:02:04.909029 20445 consensus_peers.cc:487] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 -> Peer e93d3d4db1114d719e7c2d6f99456785 (127.13.207.60:32895): Couldn't send request to peer e93d3d4db1114d719e7c2d6f99456785. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 07:02:04.909582 21090 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [term 1 LEARNER]: Refusing update from remote peer 109a60c497f74cb5bbb02a560ae83bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 07:02:04.910156 21039 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 07:02:04.912604 21039 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 92e4477c73b24415b051967647dac505 (127.13.207.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: NON_VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: true } } }
I20260808 07:02:04.914345 21039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "109a60c497f74cb5bbb02a560ae83bc1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: NON_VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: true } } }
I20260808 07:02:04.914554 21039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:04.914933 21235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "109a60c497f74cb5bbb02a560ae83bc1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: NON_VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: true } } }
I20260808 07:02:04.915031 21235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:04.915235 21090 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 92e4477c73b24415b051967647dac505 (127.13.207.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: NON_VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: true } } }
I20260808 07:02:04.916373 21119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "109a60c497f74cb5bbb02a560ae83bc1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: NON_VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: true } } }
I20260808 07:02:04.916478 21119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:02:04.923501 21135 master.cc:552] Master@127.13.207.60:32895 shutdown complete.
I20260808 07:02:04.923583 21135 master_runner.cc:305] Clearing existing system tablet
I20260808 07:02:04.923743 21135 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:04.923800 21135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:04.924712 21135 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:02:04.925412 21238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:04.925618 21135 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:04.925693 21135 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:02:04.925719 21135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/wal
uuid: "e93d3d4db1114d719e7c2d6f99456785"
format_stamp: "Formatted at 2026-08-08 07:02:04 on dist-test-slave-8tdl"
server_key: "7b4a97cf0114615a2d33cdaec37af1bf"
server_key_iv: "fd7eeda3c79a5b9cb74e3aefcf1364e8"
server_key_version: "encryptionkey@0"
I20260808 07:02:04.926617 21135 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:04.928434 21135 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 07:02:04.928517 21135 log.cc:1196] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 07:02:04.928791 21135 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785: Deleting consensus metadata
I20260808 07:02:04.929088 21135 master_runner.cc:315] Copying system tablet from 127.13.207.62:35437
I20260808 07:02:04.930248 21135 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785: tablet copy: Beginning tablet copy session from remote peer at address 127.13.207.62:35437
I20260808 07:02:04.940606 21245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.930621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.0.0.1:37784)
0808 07:02:04.930922 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:04.930936 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:04.931025 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:04.931479 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:04.931481 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:04.931717 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:04.931775 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.931778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.933178 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.933179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.934076 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:04.934080 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.934876 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.934877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:04.934944 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.935705 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:04.935710 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:04.936168 (+ 458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:04.938913 (+ 2745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.938915 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.938915 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.939190 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.939429 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:04.939430 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:04.939430 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:04.939491 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:04.940100 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:04.940102 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:04.940164 (+ 62us) client_negotiation.cc:769] Sending connection context
0808 07:02:04.940237 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 07:02:04.940383 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":146,"threads_started":1}
I20260808 07:02:04.941056 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:04.930766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37784 (local address 127.13.207.62:35437)
0808 07:02:04.930831 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:04.930833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:04.931036 (+ 203us) server_negotiation.cc:407] Connection header received
0808 07:02:04.931317 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:04.931318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:04.931335 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:04.931413 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:04.931870 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.933042 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.934349 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:04.934745 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:04.934882 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:04.936328 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:04.936329 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:04.936330 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:04.936340 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:04.938766 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.939288 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.939289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.939290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.939363 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:04.939780 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:04.939781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:04.939782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:04.939988 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:04.940062 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:04.940851 (+ 789us) server_negotiation.cc:299] Negotiation successful
0808 07:02:04.940957 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 07:02:04.941390 20480 tablet_copy_service.cc:140] P 109a60c497f74cb5bbb02a560ae83bc1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e93d3d4db1114d719e7c2d6f99456785 ({username='oryx', principal='oryx/127.13.207.60@KRBTEST.COM'} at 127.0.0.1:37784)
I20260808 07:02:04.941455 20480 tablet_copy_service.cc:161] P 109a60c497f74cb5bbb02a560ae83bc1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e93d3d4db1114d719e7c2d6f99456785 at {username='oryx', principal='oryx/127.13.207.60@KRBTEST.COM'} at 127.0.0.1:37784: session id = e93d3d4db1114d719e7c2d6f99456785-00000000000000000000000000000000
I20260808 07:02:04.942349 20480 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 07:02:04.943187 21135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 07:02:04.945207 21135 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785: tablet copy: Starting download of 0 data blocks...
I20260808 07:02:04.945412 21135 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785: tablet copy: Starting download of 1 WAL segments...
I20260808 07:02:04.947602 21135 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 07:02:04.949414 20480 tablet_copy_service.cc:342] P 109a60c497f74cb5bbb02a560ae83bc1: Request end of tablet copy session e93d3d4db1114d719e7c2d6f99456785-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.207.60@KRBTEST.COM'} at 127.0.0.1:37784
I20260808 07:02:04.949520 20480 tablet_copy_service.cc:433] P 109a60c497f74cb5bbb02a560ae83bc1: ending tablet copy session e93d3d4db1114d719e7c2d6f99456785-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e93d3d4db1114d719e7c2d6f99456785
I20260808 07:02:04.953576 21135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:04.955564 21246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:04.960382 21247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:04.961331 21249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:04.961604 21135 server_base.cc:1034] running on GCE node
I20260808 07:02:04.961786 21135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:04.961987 21135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:04.963193 21135 hybrid_clock.cc:648] HybridClock initialized: now 1786172524963183 us; error 27 us; skew 500 ppm
Aug 08 07:02:04 dist-test-slave-8tdl krb5kdc[20414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172524, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:04.965114 21135 init.cc:377] Logged in from keytab as oryx/127.13.207.60@KRBTEST.COM (short username oryx)
I20260808 07:02:04.965451 21135 webserver.cc:460] Webserver started at http://127.13.207.60:44871/ using document root <none> and password file <none>
I20260808 07:02:04.965580 21135 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:04.965618 21135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:04.966652 21135 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:04.967517 21257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:04.967674 21135 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:02:04.967737 21135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/wal
uuid: "e93d3d4db1114d719e7c2d6f99456785"
format_stamp: "Formatted at 2026-08-08 07:02:04 on dist-test-slave-8tdl"
server_key: "7b4a97cf0114615a2d33cdaec37af1bf"
server_key_iv: "fd7eeda3c79a5b9cb74e3aefcf1364e8"
server_key_version: "encryptionkey@0"
I20260808 07:02:04.967808 21135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:04.995836 21135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:04.998644 21135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:05.002169 21135 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.60:32895
I20260808 07:02:05.002317 21311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.60:32895 every 8 connection(s)
I20260808 07:02:05.002563 21135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/master-2/data/info.pb
I20260808 07:02:05.003614 21312 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:02:05.004381 21312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785: Bootstrap starting.
I20260808 07:02:05.008944 21312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:02:05.011338 21312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785: Bootstrap complete.
I20260808 07:02:05.011941 21312 raft_consensus.cc:348] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: NON_VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: true } }
I20260808 07:02:05.012056 21312 raft_consensus.cc:729] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e93d3d4db1114d719e7c2d6f99456785, State: Initialized, Role: LEARNER
I20260808 07:02:05.012099 21312 consensus_queue.cc:260] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: NON_VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: true } }
I20260808 07:02:05.012650 21312 sys_catalog.cc:564] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:02:05.012670 21315 sys_catalog.cc:455] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: NON_VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: true } } }
I20260808 07:02:05.012971 21315 sys_catalog.cc:458] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [sys.catalog]: This master's current role is: LEARNER
I20260808 07:02:05.014669 21330 catalog_manager.cc:1259] Loaded cluster ID: 8804463ead1f41d69e60b49669459c0f
I20260808 07:02:05.014848 21330 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785: loading cluster ID for follower catalog manager: success
I20260808 07:02:05.016271 21330 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785: acquiring CA information for follower catalog manager: success
I20260808 07:02:05.016629 21330 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 07:02:05.016990 21231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:05.017212 21231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:05.017243 21231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:05.018893 21231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:05.018946 21231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:05.018985 21231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:02:05.020926 21231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.1.keytab
--principal=oryx/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:45561
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=45655
--tserver_master_addrs=127.13.207.62:35437,127.13.207.61:38765,127.13.207.60:32895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:05.021184 21231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:05.021452 21231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:02:05.027053 20798 heartbeater.cc:503] Master 127.13.207.62:35437 was elected leader, sending a full tablet report...
W20260808 07:02:05.028400 21335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:05.028513 21332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:05.029264 21333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:05.029423 21231 server_base.cc:1034] running on GCE node
I20260808 07:02:05.029922 21231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:05.030298 21231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:05.031528 21231 hybrid_clock.cc:648] HybridClock initialized: now 1786172525031512 us; error 33 us; skew 500 ppm
Aug 08 07:02:05 dist-test-slave-8tdl krb5kdc[20414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172525, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:05.035292 21231 init.cc:377] Logged in from keytab as oryx/127.13.207.1@KRBTEST.COM (short username oryx)
I20260808 07:02:05.035936 21231 webserver.cc:460] Webserver started at http://127.13.207.1:45655/ using document root <none> and password file <none>
I20260808 07:02:05.036289 21231 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:05.036403 21231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:05.038275 21231 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:02:05.039108 21342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:05.039674 21231 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:02:05.039815 21231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "76046ba280e44aba95b3074f9a8701d1"
format_stamp: "Formatted at 2026-08-08 07:02:03 on dist-test-slave-8tdl"
server_key: "e2378cccf094eba97a0ec29df4f1ef9b"
server_key_iv: "179a95fb08361fc259ffc11f9f01bb87"
server_key_version: "encryptionkey@0"
I20260808 07:02:05.040362 21231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:05.079713 21231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:05.082441 21231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:05.082866 21231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:05.083202 21231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:05.083755 21231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:05.083806 21231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:05.083833 21231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:05.083846 21231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:05.091141 21231 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:45561
I20260808 07:02:05.091228 21460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:45561 every 8 connection(s)
I20260808 07:02:05.091641 21231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:02:05.093809 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 21231
I20260808 07:02:05.094044 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 20663
I20260808 07:02:05.104249 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:38057
--local_ip_for_outbound_sockets=127.13.207.2
--tserver_master_addrs=127.13.207.62:35437
--webserver_port=44175
--webserver_interface=127.13.207.2
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.2.keytab
--principal=oryx/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.207.62:35437,127.13.207.61:38765
--tserver_master_addrs=127.13.207.62:35437,127.13.207.61:38765,127.13.207.60:32895 with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:02:05 dist-test-slave-8tdl krb5kdc[20414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172525, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM
Aug 08 07:02:05 dist-test-slave-8tdl krb5kdc[20414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172525, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.61@KRBTEST.COM
Aug 08 07:02:05 dist-test-slave-8tdl krb5kdc[20414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172525, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.60@KRBTEST.COM
I20260808 07:02:05.113883 21132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.092614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:57137 (local address 127.13.207.61:38765)
0808 07:02:05.092708 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:05.092711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:05.092877 (+ 166us) server_negotiation.cc:407] Connection header received
0808 07:02:05.093811 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:05.093812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:05.093825 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:05.093881 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:05.100076 (+ 6195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.101005 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.103057 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.103359 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.103418 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.108727 (+ 5309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:05.108729 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:05.108747 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:05.108756 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:05.111408 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.112416 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.112418 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.112418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.112488 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.112880 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.112881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.112881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.113061 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:05.113154 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:05.113669 (+ 515us) server_negotiation.cc:299] Negotiation successful
0808 07:02:05.113773 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 07:02:05.114053 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.092943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:39447 (local address 127.13.207.62:35437)
0808 07:02:05.093029 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:05.093031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:05.093060 (+ 29us) server_negotiation.cc:407] Connection header received
0808 07:02:05.093855 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:05.093856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:05.093871 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:05.093921 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:05.099770 (+ 5849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.100767 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.102970 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.103356 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.103424 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.107507 (+ 4083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:05.107508 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:05.107509 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:05.107520 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:05.111419 (+ 3899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.112566 (+ 1147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.112568 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.112569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.112653 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.112921 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.112922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.112922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.113225 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:05.113349 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:05.113780 (+ 431us) server_negotiation.cc:299] Negotiation successful
0808 07:02:05.113887 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 07:02:05.114116 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.093088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:32895 (local address 127.13.207.1:59485)
0808 07:02:05.093199 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:05.093208 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:05.093709 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:05.094061 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:05.094063 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:05.099187 (+ 5124us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:05.099580 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.099592 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.101173 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.101173 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.102836 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.102841 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.103506 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.103507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.103686 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.104739 (+ 1053us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:05.104744 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:05.108462 (+ 3718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:05.111503 (+ 3041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:05.111521 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:05.111525 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:05.112280 (+ 755us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:05.112576 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:05.112576 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:05.112577 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:05.112777 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:05.113192 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:05.113201 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:05.113420 (+ 219us) client_negotiation.cc:769] Sending connection context
0808 07:02:05.113555 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 07:02:05.113788 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":66,"thread_start_us":69,"threads_started":1}
I20260808 07:02:05.114588 21467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.092830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.13.207.1:39447)
0808 07:02:05.092971 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:05.093008 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:05.093679 (+ 671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:05.094042 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:05.094047 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:05.099194 (+ 5147us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:05.099580 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.099592 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.100963 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.100965 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.102836 (+ 1871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.102841 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.103507 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.103509 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.103685 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.104616 (+ 931us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:05.104628 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:05.107296 (+ 2668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:05.111549 (+ 4253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:05.111551 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:05.111551 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:05.112290 (+ 739us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:05.112760 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:05.112761 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:05.112761 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:05.112840 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:05.113398 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:05.113399 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:05.113454 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 07:02:05.113650 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 07:02:05.114470 (+ 820us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":803,"thread_start_us":62,"threads_started":1}
I20260808 07:02:05.115136 21462 heartbeater.cc:347] Connected to a master server at 127.13.207.62:35437
I20260808 07:02:05.115231 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.093293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:59485 (local address 127.13.207.60:32895)
0808 07:02:05.093421 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:05.093424 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:05.093433 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:02:05.093866 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:05.093868 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:05.093885 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:05.094402 (+ 517us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:05.099779 (+ 5377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.101051 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.102968 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.103394 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.103644 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.108665 (+ 5021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:05.108669 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:05.108670 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:05.108684 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:05.111347 (+ 2663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.112402 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.112403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.112404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.112488 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.112881 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.112882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.112882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.113061 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:05.114929 (+ 1868us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:05.114986 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 07:02:05.115099 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":55,"threads_started":1}
I20260808 07:02:05.115291 21462 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:05.115520 21462 heartbeater.cc:511] Master 127.13.207.62:35437 requested a full tablet report, sending...
I20260808 07:02:05.115619 21466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.092526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:38765 (local address 127.13.207.1:57137)
0808 07:02:05.092700 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:05.092751 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:05.093679 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:05.094042 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:05.094047 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:05.099878 (+ 5831us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:05.099944 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.099947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.101173 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.101173 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.102958 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.102960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.103510 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.103510 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.103991 (+ 481us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.104757 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:05.104762 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:05.108446 (+ 3684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:05.111511 (+ 3065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:05.111521 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:05.111525 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:05.112280 (+ 755us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:05.112572 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:05.112573 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:05.112573 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:05.112777 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:05.113203 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:05.113203 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:05.113420 (+ 217us) client_negotiation.cc:769] Sending connection context
0808 07:02:05.113555 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 07:02:05.115050 (+ 1495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":2051,"thread_start_us":74,"threads_started":1}
I20260808 07:02:05.115767 21461 heartbeater.cc:347] Connected to a master server at 127.13.207.60:32895
I20260808 07:02:05.115824 21461 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:05.115907 21461 heartbeater.cc:511] Master 127.13.207.60:32895 requested a full tablet report, sending...
I20260808 07:02:05.116104 20460 ts_manager.cc:194] Re-registered known tserver with Master: 76046ba280e44aba95b3074f9a8701d1 (127.13.207.1:45561)
I20260808 07:02:05.116520 20460 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:39447
I20260808 07:02:05.116670 21274 ts_manager.cc:194] Registered new tserver with Master: 76046ba280e44aba95b3074f9a8701d1 (127.13.207.1:45561)
I20260808 07:02:05.117254 21463 heartbeater.cc:347] Connected to a master server at 127.13.207.61:38765
I20260808 07:02:05.117326 21463 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:05.117489 21463 heartbeater.cc:511] Master 127.13.207.61:38765 requested a full tablet report, sending...
I20260808 07:02:05.118105 21080 ts_manager.cc:194] Registered new tserver with Master: 76046ba280e44aba95b3074f9a8701d1 (127.13.207.1:45561)
W20260808 07:02:05.204353 21471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:05.204751 21471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:05.204825 21471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:05.206303 21471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:05.206369 21471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:05.206403 21471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:02:05.208248 21471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.2.keytab
--principal=oryx/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:38057
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=44175
--tserver_master_addrs=127.13.207.62:35437,127.13.207.61:38765,127.13.207.60:32895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:05.208638 21471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:05.208915 21471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:05.215512 21479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:05.215523 21477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:05.216246 21471 server_base.cc:1034] running on GCE node
W20260808 07:02:05.215534 21476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:05.216867 21471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:05.217303 21471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:05.218518 21471 hybrid_clock.cc:648] HybridClock initialized: now 1786172525218500 us; error 85 us; skew 500 ppm
Aug 08 07:02:05 dist-test-slave-8tdl krb5kdc[20414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172525, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:05.226480 21471 init.cc:377] Logged in from keytab as oryx/127.13.207.2@KRBTEST.COM (short username oryx)
I20260808 07:02:05.227607 21471 webserver.cc:460] Webserver started at http://127.13.207.2:44175/ using document root <none> and password file <none>
I20260808 07:02:05.227999 21471 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:05.228094 21471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:05.230232 21471 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:02:05.231348 21487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:05.231693 21471 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:02:05.231798 21471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "5426123c2e6b448d89af9dbe4b4c9a86"
format_stamp: "Formatted at 2026-08-08 07:02:03 on dist-test-slave-8tdl"
server_key: "3b61ac07d734998819c95dfc7300786f"
server_key_iv: "fd024b3d75e858a47613170887ce3698"
server_key_version: "encryptionkey@0"
I20260808 07:02:05.232439 21471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:05.233057 20937 heartbeater.cc:503] Master 127.13.207.62:35437 was elected leader, sending a full tablet report...
I20260808 07:02:05.245422 21471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:05.248739 21471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:05.248893 21471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:05.249245 21471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:05.249656 21471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:05.249696 21471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:05.249722 21471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:05.249763 21471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:05.259142 21471 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:38057
I20260808 07:02:05.259362 21604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:38057 every 8 connection(s)
I20260808 07:02:05.259686 21471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:02:05.263957 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 21471
I20260808 07:02:05.264190 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 20803
Aug 08 07:02:05 dist-test-slave-8tdl krb5kdc[20414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172525, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM
I20260808 07:02:05.272228 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:46325
--local_ip_for_outbound_sockets=127.13.207.3
--tserver_master_addrs=127.13.207.62:35437
--webserver_port=38005
--webserver_interface=127.13.207.3
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.3.keytab
--principal=oryx/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.207.62:35437,127.13.207.61:38765
--tserver_master_addrs=127.13.207.62:35437,127.13.207.61:38765,127.13.207.60:32895 with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:02:05 dist-test-slave-8tdl krb5kdc[20414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172525, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.61@KRBTEST.COM
Aug 08 07:02:05 dist-test-slave-8tdl krb5kdc[20414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172525, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.60@KRBTEST.COM
I20260808 07:02:05.277647 21132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.261162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:45447 (local address 127.13.207.61:38765)
0808 07:02:05.261208 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:05.261210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:05.261415 (+ 205us) server_negotiation.cc:407] Connection header received
0808 07:02:05.261697 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:05.261699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:05.261728 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:05.261785 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:05.263080 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.263682 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.268394 (+ 4712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.268782 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.268837 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.273461 (+ 4624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:05.273463 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:05.273464 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:05.273476 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:05.275476 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.276029 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.276031 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.276032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.276115 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.276734 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.276735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.276735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.276925 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:05.276984 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:05.277392 (+ 408us) server_negotiation.cc:299] Negotiation successful
0808 07:02:05.277494 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:02:05.277937 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.261289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:59149 (local address 127.13.207.62:35437)
0808 07:02:05.261338 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:05.261340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:05.261840 (+ 500us) server_negotiation.cc:407] Connection header received
0808 07:02:05.261947 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:05.261948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:05.261961 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:05.262032 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:05.263100 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.263983 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.268386 (+ 4403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.268754 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.269013 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.273201 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:05.273203 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:05.273204 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:05.273214 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:05.275490 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.276359 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.276360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.276360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.276434 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.277265 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.277266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.277267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.277424 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:05.277512 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:05.277718 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 07:02:05.277815 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:02:05.278355 21611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.261053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:38765 (local address 127.13.207.2:45447)
0808 07:02:05.261335 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:05.261348 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:05.261592 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:05.262529 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:05.262529 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:05.262721 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:05.262950 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.262956 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.263861 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.263862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.268181 (+ 4319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.268186 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.269045 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.269046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.269221 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.270226 (+ 1005us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:05.270239 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:05.273027 (+ 2788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:05.275579 (+ 2552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:05.275594 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:05.275598 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:05.275939 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:05.276205 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:05.276206 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:05.276206 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:05.276639 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:05.277024 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:05.277033 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:05.277223 (+ 190us) client_negotiation.cc:769] Sending connection context
0808 07:02:05.277804 (+ 581us) client_negotiation.cc:240] Negotiation successful
0808 07:02:05.278039 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":65,"threads_started":1}
I20260808 07:02:05.278367 21610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.261214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.13.207.2:59149)
0808 07:02:05.261776 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:05.261786 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:05.261867 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:05.262106 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:05.262108 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:05.262600 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:05.262950 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.262958 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.264094 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.264095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.268179 (+ 4084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.268185 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.269037 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.269039 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.269221 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.270410 (+ 1189us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:05.270415 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:05.273027 (+ 2612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:05.276025 (+ 2998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:05.276026 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:05.276026 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:05.276251 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:05.277084 (+ 833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:05.277084 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:05.277085 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:05.277176 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:05.277546 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:05.277547 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:05.277589 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 07:02:05.278072 (+ 483us) client_negotiation.cc:240] Negotiation successful
0808 07:02:05.278194 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"mutex_wait_us":144,"thread_start_us":65,"threads_started":1}
I20260808 07:02:05.278991 21606 heartbeater.cc:347] Connected to a master server at 127.13.207.62:35437
I20260808 07:02:05.279090 21606 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:05.279345 21606 heartbeater.cc:511] Master 127.13.207.62:35437 requested a full tablet report, sending...
I20260808 07:02:05.279805 20460 ts_manager.cc:194] Re-registered known tserver with Master: 5426123c2e6b448d89af9dbe4b4c9a86 (127.13.207.2:38057)
I20260808 07:02:05.279839 21607 heartbeater.cc:347] Connected to a master server at 127.13.207.61:38765
I20260808 07:02:05.279881 21607 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:05.279987 21607 heartbeater.cc:511] Master 127.13.207.61:38765 requested a full tablet report, sending...
I20260808 07:02:05.280301 21080 ts_manager.cc:194] Registered new tserver with Master: 5426123c2e6b448d89af9dbe4b4c9a86 (127.13.207.2:38057)
I20260808 07:02:05.280354 20460 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:59149
I20260808 07:02:05.281136 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.260709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:42435 (local address 127.13.207.60:32895)
0808 07:02:05.260769 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:05.260772 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:05.261004 (+ 232us) server_negotiation.cc:407] Connection header received
0808 07:02:05.261708 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:05.261709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:05.261725 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:05.261783 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:05.263096 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.263979 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.272224 (+ 8245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.272577 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.272657 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.275397 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:05.275399 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:05.275400 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:05.275410 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:05.276970 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.277387 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.277388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.277388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.277444 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.277651 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.277652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.277652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.277784 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:05.277845 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:05.280872 (+ 3027us) server_negotiation.cc:299] Negotiation successful
0808 07:02:05.280997 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:02:05.281435 21609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.260671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:32895 (local address 127.13.207.2:42435)
0808 07:02:05.260852 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:05.260895 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:05.261572 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:05.261862 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:05.261867 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:05.262600 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:05.262950 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.262958 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.264094 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.264095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.272089 (+ 7994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.272094 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.272727 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.272728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.272859 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.273565 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:05.273570 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:05.275281 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:05.277067 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:05.277068 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:05.277068 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:05.277306 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:05.277517 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:05.277517 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:05.277518 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:05.277579 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:05.280128 (+ 2549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:05.280129 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:05.280706 (+ 577us) client_negotiation.cc:769] Sending connection context
0808 07:02:05.280780 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 07:02:05.280898 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":6991,"thread_start_us":93,"threads_started":1}
I20260808 07:02:05.281950 21605 heartbeater.cc:347] Connected to a master server at 127.13.207.60:32895
I20260808 07:02:05.282024 21605 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:05.282119 21605 heartbeater.cc:511] Master 127.13.207.60:32895 requested a full tablet report, sending...
I20260808 07:02:05.282763 21274 ts_manager.cc:194] Registered new tserver with Master: 5426123c2e6b448d89af9dbe4b4c9a86 (127.13.207.2:38057)
I20260808 07:02:05.347368 21038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.343460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:32895 (local address 127.0.0.1:36806)
0808 07:02:05.343524 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:05.343536 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:05.343582 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:05.343892 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:05.343893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:05.343896 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:05.344141 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.344146 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.345242 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.345243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.346175 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.346179 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.347092 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.347093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.347158 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.347162 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:02:05.347218 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 07:02:05.347267 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 07:02:05.347393 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.343632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36806 (local address 127.13.207.60:32895)
0808 07:02:05.343712 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:05.343714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:05.343724 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:02:05.343754 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:05.343755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:05.343779 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:05.343849 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:05.344252 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.345052 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.346285 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.346962 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.347047 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.347115 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:05.347241 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 07:02:05.347283 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 07:02:05.347746 21285 raft_consensus.cc:1203] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 07:02:05.347954 21315 sys_catalog.cc:455] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 109a60c497f74cb5bbb02a560ae83bc1. Latest consensus state: current_term: 1 leader_uuid: "109a60c497f74cb5bbb02a560ae83bc1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: NON_VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: true } } }
I20260808 07:02:05.348088 21315 sys_catalog.cc:458] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [sys.catalog]: This master's current role is: LEARNER
W20260808 07:02:05.371249 21612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:05.371497 21612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:05.371527 21612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:05.372905 21612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:05.372965 21612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:05.372997 21612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:02:05.374835 21612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.3.keytab
--principal=oryx/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:46325
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=38005
--tserver_master_addrs=127.13.207.62:35437,127.13.207.61:38765,127.13.207.60:32895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:05.375135 21612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:05.375402 21612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:05.381255 21621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:05.381256 21617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:05.381260 21618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:05.381599 21612 server_base.cc:1034] running on GCE node
I20260808 07:02:05.381852 21612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:05.382097 21612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:05.383275 21612 hybrid_clock.cc:648] HybridClock initialized: now 1786172525383233 us; error 55 us; skew 500 ppm
Aug 08 07:02:05 dist-test-slave-8tdl krb5kdc[20414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172525, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:05.386329 21612 init.cc:377] Logged in from keytab as oryx/127.13.207.3@KRBTEST.COM (short username oryx)
I20260808 07:02:05.387001 21612 webserver.cc:460] Webserver started at http://127.13.207.3:38005/ using document root <none> and password file <none>
I20260808 07:02:05.387265 21612 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:05.387331 21612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:05.389035 21612 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:02:05.389894 21628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:05.390161 21612 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:05.390295 21612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "acb5dce36f6043409bc52171d9ba692f"
format_stamp: "Formatted at 2026-08-08 07:02:04 on dist-test-slave-8tdl"
server_key: "a119febd3a084cefbe62674159a01cbc"
server_key_iv: "71c4942d0a68177a01fae605c9fd82c9"
server_key_version: "encryptionkey@0"
I20260808 07:02:05.390787 21612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:05.414469 21612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:05.417366 21612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:05.417564 21612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:05.417865 21612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:05.418248 21612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:05.418309 21612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:05.418354 21612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:05.418380 21612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:05.426592 21612 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:46325
I20260808 07:02:05.426666 21747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:46325 every 8 connection(s)
I20260808 07:02:05.427171 21612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:02:05.432156 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 21612
Aug 08 07:02:05 dist-test-slave-8tdl krb5kdc[20414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172525, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.61@KRBTEST.COM
Aug 08 07:02:05 dist-test-slave-8tdl krb5kdc[20414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172525, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.60@KRBTEST.COM
Aug 08 07:02:05 dist-test-slave-8tdl krb5kdc[20414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172525, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM
I20260808 07:02:05.441813 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.428421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:51895 (local address 127.13.207.60:32895)
0808 07:02:05.428472 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:05.428474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:05.429046 (+ 572us) server_negotiation.cc:407] Connection header received
0808 07:02:05.429578 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:05.429579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:05.429596 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:05.429629 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:05.430919 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.431702 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.434382 (+ 2680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.434710 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.434757 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.438650 (+ 3893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:05.438652 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:05.438653 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:05.438661 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:05.440343 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.440779 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.440780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.440781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.440841 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.441020 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.441021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.441021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.441160 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:05.441218 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:05.441599 (+ 381us) server_negotiation.cc:299] Negotiation successful
0808 07:02:05.441689 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:02:05.441911 21132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.428586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:49875 (local address 127.13.207.61:38765)
0808 07:02:05.428630 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:05.428632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:05.428899 (+ 267us) server_negotiation.cc:407] Connection header received
0808 07:02:05.429458 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:05.429459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:05.429473 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:05.429527 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:05.430850 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.431922 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.433698 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.434127 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.434174 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.437792 (+ 3618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:05.437794 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:05.437795 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:05.437804 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:05.440018 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.440608 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.440609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.440609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.440674 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.440996 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.440997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.440998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.441150 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:05.441212 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:05.441707 (+ 495us) server_negotiation.cc:299] Negotiation successful
0808 07:02:05.441801 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:02:05.442584 21753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.428339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:32895 (local address 127.13.207.3:51895)
0808 07:02:05.428696 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:05.428711 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:05.429442 (+ 731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:05.429703 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:05.429704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:05.430602 (+ 898us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:05.430737 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.430740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.433201 (+ 2461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.433201 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.434272 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.434274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.434865 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.434866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.434950 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.435449 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:05.435454 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:05.438431 (+ 2977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:05.440449 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:05.440450 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:05.440450 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:05.440673 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:05.440904 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:05.440905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:05.440906 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:05.440959 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:05.441244 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:05.441253 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:05.441438 (+ 185us) client_negotiation.cc:769] Sending connection context
0808 07:02:05.441968 (+ 530us) client_negotiation.cc:240] Negotiation successful
0808 07:02:05.442243 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":8,"thread_start_us":71,"threads_started":1}
I20260808 07:02:05.442587 21754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.428569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:38765 (local address 127.13.207.3:49875)
0808 07:02:05.428814 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:05.428825 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:05.429331 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:05.429619 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:05.429622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:05.430374 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:05.430596 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.430603 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.432092 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.432094 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.433411 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.433416 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.434344 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.434346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.434433 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.435120 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:05.435134 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:05.437630 (+ 2496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:05.440126 (+ 2496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:05.440140 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:05.440144 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:05.440515 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:05.440739 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:05.440740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:05.440740 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:05.440914 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:05.441242 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:05.441249 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:05.441525 (+ 276us) client_negotiation.cc:769] Sending connection context
0808 07:02:05.442051 (+ 526us) client_negotiation.cc:240] Negotiation successful
0808 07:02:05.442315 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":209,"thread_start_us":178,"threads_started":1}
I20260808 07:02:05.443186 21750 heartbeater.cc:347] Connected to a master server at 127.13.207.61:38765
I20260808 07:02:05.443264 21748 heartbeater.cc:347] Connected to a master server at 127.13.207.60:32895
I20260808 07:02:05.443298 21750 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:05.443334 21748 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:05.443536 21750 heartbeater.cc:511] Master 127.13.207.61:38765 requested a full tablet report, sending...
I20260808 07:02:05.443538 21748 heartbeater.cc:511] Master 127.13.207.60:32895 requested a full tablet report, sending...
I20260808 07:02:05.444123 21080 ts_manager.cc:194] Registered new tserver with Master: acb5dce36f6043409bc52171d9ba692f (127.13.207.3:46325)
I20260808 07:02:05.446157 21274 ts_manager.cc:194] Registered new tserver with Master: acb5dce36f6043409bc52171d9ba692f (127.13.207.3:46325)
I20260808 07:02:05.447734 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.428065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:40329 (local address 127.13.207.62:35437)
0808 07:02:05.428122 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:05.428125 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:05.428753 (+ 628us) server_negotiation.cc:407] Connection header received
0808 07:02:05.429571 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:05.429573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:05.429590 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:05.429628 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:05.430854 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.431662 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.433647 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.433967 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.434021 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.443468 (+ 9447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:05.443471 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:05.443472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:05.443483 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:05.445868 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.446341 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.446342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.446343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.446416 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.446653 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.446654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.446655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.446858 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:05.446914 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:05.447479 (+ 565us) server_negotiation.cc:299] Negotiation successful
0808 07:02:05.447596 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:02:05.448179 21752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.428045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.13.207.3:40329)
0808 07:02:05.428225 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:05.428266 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:05.429442 (+ 1176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:05.429703 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:05.429704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:05.430582 (+ 878us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:05.430737 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.430740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.431847 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.431847 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.433439 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.433443 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.434118 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.434119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.434279 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.436212 (+ 1933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:05.436217 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:05.443316 (+ 7099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:05.445990 (+ 2674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:05.445992 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:05.445992 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:05.446237 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:05.446501 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:05.446502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:05.446502 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:05.446573 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:05.446973 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:05.446974 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:05.447339 (+ 365us) client_negotiation.cc:769] Sending connection context
0808 07:02:05.447870 (+ 531us) client_negotiation.cc:240] Negotiation successful
0808 07:02:05.448025 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":40,"thread_start_us":93,"threads_started":1}
I20260808 07:02:05.448882 21749 heartbeater.cc:347] Connected to a master server at 127.13.207.62:35437
I20260808 07:02:05.448952 21749 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:05.449038 21749 heartbeater.cc:511] Master 127.13.207.62:35437 requested a full tablet report, sending...
I20260808 07:02:05.449555 21761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.434400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36496 (local address 127.13.207.61:38765)
0808 07:02:05.434820 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:05.434822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:05.438031 (+ 3209us) server_negotiation.cc:407] Connection header received
0808 07:02:05.438063 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:05.438064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:05.438081 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:05.438346 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:05.439208 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.440037 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.441191 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.441475 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.441559 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.444392 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:05.444393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:05.444393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:05.444401 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:05.446723 (+ 2322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.447124 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.447125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.447125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.447224 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.447490 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.447491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.447492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.447686 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:05.448059 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:05.449303 (+ 1244us) server_negotiation.cc:299] Negotiation successful
0808 07:02:05.449415 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":78,"threads_started":1}
I20260808 07:02:05.449496 20460 ts_manager.cc:194] Re-registered known tserver with Master: acb5dce36f6043409bc52171d9ba692f (127.13.207.3:46325)
I20260808 07:02:05.450107 20460 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:40329
I20260808 07:02:05.450347 21759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.433378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37800 (local address 127.13.207.62:35437)
0808 07:02:05.435930 (+ 2552us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:05.435933 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:05.435954 (+ 21us) server_negotiation.cc:407] Connection header received
0808 07:02:05.435983 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:05.435983 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:05.435998 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:05.436071 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:05.437054 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.437584 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.444608 (+ 7024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.445013 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.445101 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.446662 (+ 1561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:05.446664 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:05.446664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:05.446720 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:05.448959 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.449405 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.449406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.449407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.449470 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.449689 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.449690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.449690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.449821 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:05.449905 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:05.450131 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 07:02:05.450230 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2520,"thread_start_us":65,"threads_started":1}
I20260808 07:02:05.451279 21764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.434957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36816 (local address 127.13.207.60:32895)
0808 07:02:05.435363 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:05.435366 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:05.439356 (+ 3990us) server_negotiation.cc:407] Connection header received
0808 07:02:05.439387 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:05.439389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:05.439404 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:05.439474 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:05.440249 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.441003 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.442268 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.442591 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.442666 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.447395 (+ 4729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:05.447397 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:05.447398 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:05.447406 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:05.449772 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.450271 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.450273 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.450273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.450346 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.450545 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.450545 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.450546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.450725 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:05.450825 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:05.450942 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 07:02:05.451163 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":71,"threads_started":1}
I20260808 07:02:05.453073 20460 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:02:05.454877 20460 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:02:05.455861 21767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "109a60c497f74cb5bbb02a560ae83bc1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: NON_VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: true } } }
I20260808 07:02:05.455999 21767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:05.457989 21767 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1: attempting to promote NON_VOTER e93d3d4db1114d719e7c2d6f99456785 to VOTER
I20260808 07:02:05.458173 21767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: false } }
I20260808 07:02:05.458745 21285 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [term 1 LEARNER]: Refusing update from remote peer 109a60c497f74cb5bbb02a560ae83bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 07:02:05.458797 21090 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [term 1 FOLLOWER]: Refusing update from remote peer 109a60c497f74cb5bbb02a560ae83bc1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 07:02:05.459019 21236 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 07:02:05.459151 21236 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 07:02:05.461946 21766 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, e93d3d4db1114d719e7c2d6f99456785 (127.13.207.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: false } } }
I20260808 07:02:05.462147 21285 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, e93d3d4db1114d719e7c2d6f99456785 (127.13.207.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: false } } }
I20260808 07:02:05.462989 21090 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, e93d3d4db1114d719e7c2d6f99456785 (127.13.207.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: false } } }
I20260808 07:02:05.463124 21766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "109a60c497f74cb5bbb02a560ae83bc1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: false } } }
I20260808 07:02:05.463196 21766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:05.463243 21766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "109a60c497f74cb5bbb02a560ae83bc1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: false } } }
I20260808 07:02:05.463282 21766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:05.463310 21766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "109a60c497f74cb5bbb02a560ae83bc1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: false } } }
I20260808 07:02:05.463340 21766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:05.464059 21315 sys_catalog.cc:455] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "109a60c497f74cb5bbb02a560ae83bc1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: false } } }
I20260808 07:02:05.464178 21315 sys_catalog.cc:458] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:02:05.465912 21775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "109a60c497f74cb5bbb02a560ae83bc1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: false } } }
I20260808 07:02:05.466053 21775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:02:05.471118 21038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.466341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:38057 (local address 127.0.0.1:40658)
0808 07:02:05.466398 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:05.466409 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:05.466449 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:05.467096 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:05.467097 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:05.467099 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:05.467164 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.467166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.468614 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.468615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.469795 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.469798 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.470860 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.470861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.470929 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.470934 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:02:05.470973 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 07:02:05.471023 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 07:02:05.471737 21776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.466573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40658 (local address 127.13.207.2:38057)
0808 07:02:05.466876 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:05.466882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:05.466893 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:02:05.466941 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:05.466943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:05.466972 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:05.467254 (+ 282us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:05.467325 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.468519 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.469927 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.470763 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.471470 (+ 707us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.471575 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:05.471605 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 07:02:05.471654 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":143,"threads_started":1}
I20260808 07:02:05.472653 21537 tablet_service.cc:1467] Processing CreateTablet for tablet 6c55b358933944eeb406288477bc84ff (DEFAULT_TABLE table=test-table [id=c0a69b1d75c64f928d82cc445f4fd49e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:05.473084 21537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c55b358933944eeb406288477bc84ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:05.473270 21777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.466877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:45561 (local address 127.0.0.1:36580)
0808 07:02:05.467056 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:05.467071 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:05.467165 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:05.467998 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:05.467999 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:05.468002 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:05.468093 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.468096 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.470244 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.470245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.472024 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.472027 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.472994 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.472995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.473067 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.473072 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:02:05.473136 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 07:02:05.473187 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
I20260808 07:02:05.473284 21779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.467354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36580 (local address 127.13.207.1:45561)
0808 07:02:05.467816 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:05.467821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:05.467832 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:02:05.467870 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:05.467872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:05.467887 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:05.467971 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:05.468277 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.470117 (+ 1840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.472146 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.472862 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.472992 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.473072 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:05.473152 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 07:02:05.473193 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":117,"threads_started":1}
I20260808 07:02:05.473929 21393 tablet_service.cc:1467] Processing CreateTablet for tablet 6c55b358933944eeb406288477bc84ff (DEFAULT_TABLE table=test-table [id=c0a69b1d75c64f928d82cc445f4fd49e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:05.474346 21393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c55b358933944eeb406288477bc84ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:05.477811 21781 tablet_bootstrap.cc:492] T 6c55b358933944eeb406288477bc84ff P 76046ba280e44aba95b3074f9a8701d1: Bootstrap starting.
I20260808 07:02:05.478161 21782 tablet_bootstrap.cc:492] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86: Bootstrap starting.
I20260808 07:02:05.478382 21778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.467058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:46325 (local address 127.0.0.1:51936)
0808 07:02:05.468059 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:05.468069 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:05.468126 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:05.468534 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:05.468535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:05.468537 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:05.468586 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.468587 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.469898 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.469899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.471012 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.471014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.472078 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.472079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.477970 (+ 5891us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.478034 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 07:02:05.478129 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 07:02:05.478207 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":979,"thread_start_us":67,"threads_started":1}
I20260808 07:02:05.478936 21781 tablet_bootstrap.cc:654] T 6c55b358933944eeb406288477bc84ff P 76046ba280e44aba95b3074f9a8701d1: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:05.478937 21782 tablet_bootstrap.cc:654] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:05.479048 21780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.467567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51936 (local address 127.13.207.3:46325)
0808 07:02:05.467826 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:05.467832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:05.468182 (+ 350us) server_negotiation.cc:407] Connection header received
0808 07:02:05.468225 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:05.468227 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:05.468243 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:05.468460 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:05.468664 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.469791 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.471101 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.471895 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.471966 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.472041 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:05.478793 (+ 6752us) server_negotiation.cc:299] Negotiation successful
0808 07:02:05.478885 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":110,"threads_started":1}
I20260808 07:02:05.479394 21782 log.cc:824] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:05.479529 21781 log.cc:824] T 6c55b358933944eeb406288477bc84ff P 76046ba280e44aba95b3074f9a8701d1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:05.479830 21680 tablet_service.cc:1467] Processing CreateTablet for tablet 6c55b358933944eeb406288477bc84ff (DEFAULT_TABLE table=test-table [id=c0a69b1d75c64f928d82cc445f4fd49e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:05.480232 21680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c55b358933944eeb406288477bc84ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:05.480688 21782 tablet_bootstrap.cc:492] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86: No bootstrap required, opened a new log
I20260808 07:02:05.480763 21782 ts_tablet_manager.cc:1397] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:02:05.480965 21781 tablet_bootstrap.cc:492] T 6c55b358933944eeb406288477bc84ff P 76046ba280e44aba95b3074f9a8701d1: No bootstrap required, opened a new log
I20260808 07:02:05.481065 21781 ts_tablet_manager.cc:1397] T 6c55b358933944eeb406288477bc84ff P 76046ba280e44aba95b3074f9a8701d1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 07:02:05.483618 21782 raft_consensus.cc:348] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38057 } } peers { permanent_uuid: "76046ba280e44aba95b3074f9a8701d1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45561 } } peers { permanent_uuid: "acb5dce36f6043409bc52171d9ba692f" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46325 } }
I20260808 07:02:05.483825 21782 raft_consensus.cc:374] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:05.483865 21782 raft_consensus.cc:729] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5426123c2e6b448d89af9dbe4b4c9a86, State: Initialized, Role: FOLLOWER
I20260808 07:02:05.484050 21782 consensus_queue.cc:260] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38057 } } peers { permanent_uuid: "76046ba280e44aba95b3074f9a8701d1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45561 } } peers { permanent_uuid: "acb5dce36f6043409bc52171d9ba692f" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46325 } }
I20260808 07:02:05.484368 21782 ts_tablet_manager.cc:1428] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 07:02:05.484411 21606 heartbeater.cc:503] Master 127.13.207.62:35437 was elected leader, sending a full tablet report...
I20260808 07:02:05.484411 21781 raft_consensus.cc:348] T 6c55b358933944eeb406288477bc84ff P 76046ba280e44aba95b3074f9a8701d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38057 } } peers { permanent_uuid: "76046ba280e44aba95b3074f9a8701d1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45561 } } peers { permanent_uuid: "acb5dce36f6043409bc52171d9ba692f" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46325 } }
I20260808 07:02:05.484750 21781 raft_consensus.cc:374] T 6c55b358933944eeb406288477bc84ff P 76046ba280e44aba95b3074f9a8701d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:05.484800 21781 raft_consensus.cc:729] T 6c55b358933944eeb406288477bc84ff P 76046ba280e44aba95b3074f9a8701d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76046ba280e44aba95b3074f9a8701d1, State: Initialized, Role: FOLLOWER
I20260808 07:02:05.484936 21781 consensus_queue.cc:260] T 6c55b358933944eeb406288477bc84ff P 76046ba280e44aba95b3074f9a8701d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38057 } } peers { permanent_uuid: "76046ba280e44aba95b3074f9a8701d1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45561 } } peers { permanent_uuid: "acb5dce36f6043409bc52171d9ba692f" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46325 } }
I20260808 07:02:05.485240 21462 heartbeater.cc:503] Master 127.13.207.62:35437 was elected leader, sending a full tablet report...
I20260808 07:02:05.485579 21781 ts_tablet_manager.cc:1428] T 6c55b358933944eeb406288477bc84ff P 76046ba280e44aba95b3074f9a8701d1: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 07:02:05.485807 21786 tablet_bootstrap.cc:492] T 6c55b358933944eeb406288477bc84ff P acb5dce36f6043409bc52171d9ba692f: Bootstrap starting.
I20260808 07:02:05.486583 21786 tablet_bootstrap.cc:654] T 6c55b358933944eeb406288477bc84ff P acb5dce36f6043409bc52171d9ba692f: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:05.486963 21786 log.cc:824] T 6c55b358933944eeb406288477bc84ff P acb5dce36f6043409bc52171d9ba692f: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:05.488045 21786 tablet_bootstrap.cc:492] T 6c55b358933944eeb406288477bc84ff P acb5dce36f6043409bc52171d9ba692f: No bootstrap required, opened a new log
I20260808 07:02:05.488147 21786 ts_tablet_manager.cc:1397] T 6c55b358933944eeb406288477bc84ff P acb5dce36f6043409bc52171d9ba692f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:02:05.490068 21786 raft_consensus.cc:348] T 6c55b358933944eeb406288477bc84ff P acb5dce36f6043409bc52171d9ba692f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38057 } } peers { permanent_uuid: "76046ba280e44aba95b3074f9a8701d1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45561 } } peers { permanent_uuid: "acb5dce36f6043409bc52171d9ba692f" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46325 } }
I20260808 07:02:05.490252 21786 raft_consensus.cc:374] T 6c55b358933944eeb406288477bc84ff P acb5dce36f6043409bc52171d9ba692f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:05.490302 21786 raft_consensus.cc:729] T 6c55b358933944eeb406288477bc84ff P acb5dce36f6043409bc52171d9ba692f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb5dce36f6043409bc52171d9ba692f, State: Initialized, Role: FOLLOWER
I20260808 07:02:05.490438 21786 consensus_queue.cc:260] T 6c55b358933944eeb406288477bc84ff P acb5dce36f6043409bc52171d9ba692f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38057 } } peers { permanent_uuid: "76046ba280e44aba95b3074f9a8701d1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45561 } } peers { permanent_uuid: "acb5dce36f6043409bc52171d9ba692f" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46325 } }
I20260808 07:02:05.490975 21786 ts_tablet_manager.cc:1428] T 6c55b358933944eeb406288477bc84ff P acb5dce36f6043409bc52171d9ba692f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:02:05.491045 21749 heartbeater.cc:503] Master 127.13.207.62:35437 was elected leader, sending a full tablet report...
W20260808 07:02:05.510882 21608 tablet.cc:2369] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:02:05.592746 21464 tablet.cc:2369] T 6c55b358933944eeb406288477bc84ff P 76046ba280e44aba95b3074f9a8701d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:02:05.626634 21787 raft_consensus.cc:482] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:02:05.626857 21787 raft_consensus.cc:504] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38057 } } peers { permanent_uuid: "76046ba280e44aba95b3074f9a8701d1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45561 } } peers { permanent_uuid: "acb5dce36f6043409bc52171d9ba692f" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46325 } }
I20260808 07:02:05.627306 21787 leader_election.cc:288] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76046ba280e44aba95b3074f9a8701d1 (127.13.207.1:45561), acb5dce36f6043409bc52171d9ba692f (127.13.207.3:46325)
I20260808 07:02:05.631527 21609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.627517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:45561 (local address 127.13.207.2:57849)
0808 07:02:05.627590 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:05.627608 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:05.627686 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:05.627872 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:05.627872 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:05.627875 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:05.627967 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.627969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.628933 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.628935 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.630274 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.630277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.631174 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.631176 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.631269 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.631276 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 07:02:05.631344 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 07:02:05.631403 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":15}
I20260808 07:02:05.631574 21779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.627665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:57849 (local address 127.13.207.1:45561)
0808 07:02:05.627731 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:05.627734 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:05.627742 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:02:05.627776 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:05.627777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:05.627787 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:05.627849 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:05.628062 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.628848 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.630412 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.631052 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.631151 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.631163 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:05.631364 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 07:02:05.631511 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:02:05.631572 21780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.627817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:41009 (local address 127.13.207.3:46325)
0808 07:02:05.627867 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:05.627869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:05.627876 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:02:05.627902 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:05.627903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:05.627911 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:05.627962 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:05.628095 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.628824 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.630413 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.631098 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.631170 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.631178 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:05.631365 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 07:02:05.631493 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:02:05.631639 21611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.627685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:46325 (local address 127.13.207.2:41009)
0808 07:02:05.627736 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:05.627747 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:05.627790 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:05.627986 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:05.627986 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:05.627988 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:05.628037 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.628039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.628934 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.628935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.630274 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:05.630277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.631175 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.631176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:05.631269 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.631276 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 07:02:05.631344 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 07:02:05.631403 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":63}
I20260808 07:02:05.631954 21414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c55b358933944eeb406288477bc84ff" candidate_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76046ba280e44aba95b3074f9a8701d1" is_pre_election: true
I20260808 07:02:05.631983 21701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c55b358933944eeb406288477bc84ff" candidate_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb5dce36f6043409bc52171d9ba692f" is_pre_election: true
I20260808 07:02:05.632166 21414 raft_consensus.cc:2454] T 6c55b358933944eeb406288477bc84ff P 76046ba280e44aba95b3074f9a8701d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5426123c2e6b448d89af9dbe4b4c9a86 in term 0.
I20260808 07:02:05.632179 21701 raft_consensus.cc:2454] T 6c55b358933944eeb406288477bc84ff P acb5dce36f6043409bc52171d9ba692f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5426123c2e6b448d89af9dbe4b4c9a86 in term 0.
I20260808 07:02:05.632457 21490 leader_election.cc:302] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5426123c2e6b448d89af9dbe4b4c9a86, acb5dce36f6043409bc52171d9ba692f; no voters:
I20260808 07:02:05.632663 21787 raft_consensus.cc:2781] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:02:05.632766 21787 raft_consensus.cc:482] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:02:05.632794 21787 raft_consensus.cc:3037] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:05.633734 21787 raft_consensus.cc:504] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38057 } } peers { permanent_uuid: "76046ba280e44aba95b3074f9a8701d1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45561 } } peers { permanent_uuid: "acb5dce36f6043409bc52171d9ba692f" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46325 } }
I20260808 07:02:05.633890 21787 leader_election.cc:288] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86 [CANDIDATE]: Term 1 election: Requested vote from peers 76046ba280e44aba95b3074f9a8701d1 (127.13.207.1:45561), acb5dce36f6043409bc52171d9ba692f (127.13.207.3:46325)
I20260808 07:02:05.634156 21414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c55b358933944eeb406288477bc84ff" candidate_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76046ba280e44aba95b3074f9a8701d1"
I20260808 07:02:05.634256 21414 raft_consensus.cc:3037] T 6c55b358933944eeb406288477bc84ff P 76046ba280e44aba95b3074f9a8701d1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:05.634364 21701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c55b358933944eeb406288477bc84ff" candidate_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb5dce36f6043409bc52171d9ba692f"
I20260808 07:02:05.634461 21701 raft_consensus.cc:3037] T 6c55b358933944eeb406288477bc84ff P acb5dce36f6043409bc52171d9ba692f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:05.635313 21414 raft_consensus.cc:2454] T 6c55b358933944eeb406288477bc84ff P 76046ba280e44aba95b3074f9a8701d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5426123c2e6b448d89af9dbe4b4c9a86 in term 1.
I20260808 07:02:05.635591 21701 raft_consensus.cc:2454] T 6c55b358933944eeb406288477bc84ff P acb5dce36f6043409bc52171d9ba692f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5426123c2e6b448d89af9dbe4b4c9a86 in term 1.
I20260808 07:02:05.635589 21490 leader_election.cc:302] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5426123c2e6b448d89af9dbe4b4c9a86, 76046ba280e44aba95b3074f9a8701d1; no voters:
I20260808 07:02:05.635721 21787 raft_consensus.cc:2781] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:02:05.635984 21787 raft_consensus.cc:686] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86 [term 1 LEADER]: Becoming Leader. State: Replica: 5426123c2e6b448d89af9dbe4b4c9a86, State: Running, Role: LEADER
I20260808 07:02:05.636112 21787 consensus_queue.cc:237] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38057 } } peers { permanent_uuid: "76046ba280e44aba95b3074f9a8701d1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45561 } } peers { permanent_uuid: "acb5dce36f6043409bc52171d9ba692f" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46325 } }
I20260808 07:02:05.637068 20460 catalog_manager.cc:5495] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5426123c2e6b448d89af9dbe4b4c9a86 (127.13.207.2). New cstate: current_term: 1 leader_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76046ba280e44aba95b3074f9a8701d1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acb5dce36f6043409bc52171d9ba692f" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46325 } health_report { overall_health: UNKNOWN } } }
I20260808 07:02:05.655576 21776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.652489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40674 (local address 127.13.207.2:38057)
0808 07:02:05.652562 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:05.652564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:05.652578 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:02:05.652622 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:05.652623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:05.652643 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:05.652724 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:02:05.653048 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.653560 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.654759 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.655001 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.655064 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.655261 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:02:05.655372 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:02:05.655442 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:05.655479 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 07:02:05.655507 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:02:05.657967 21414 raft_consensus.cc:1261] T 6c55b358933944eeb406288477bc84ff P 76046ba280e44aba95b3074f9a8701d1 [term 1 FOLLOWER]: Refusing update from remote peer 5426123c2e6b448d89af9dbe4b4c9a86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:05.657984 21701 raft_consensus.cc:1261] T 6c55b358933944eeb406288477bc84ff P acb5dce36f6043409bc52171d9ba692f [term 1 FOLLOWER]: Refusing update from remote peer 5426123c2e6b448d89af9dbe4b4c9a86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:05.658393 21787 consensus_queue.cc:1035] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76046ba280e44aba95b3074f9a8701d1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:05.658586 21791 consensus_queue.cc:1035] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb5dce36f6043409bc52171d9ba692f" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:05.662559 21795 mvcc.cc:205] Tried to move back new op lower bound from 7316162665091276800 to 7316162665006153728. Current Snapshot: MvccSnapshot[applied={T|T < 7316162665091276800}]
I20260808 07:02:05.667317 20460 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:02:05.667546 20460 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:02:05.669780 20460 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 109a60c497f74cb5bbb02a560ae83bc1: Sending DeleteTablet for 3 replicas of tablet 6c55b358933944eeb406288477bc84ff
I20260808 07:02:05.670862 21537 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c55b358933944eeb406288477bc84ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:05 UTC) from {username='oryx', principal='oryx/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:40658
I20260808 07:02:05.671078 21393 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c55b358933944eeb406288477bc84ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:05 UTC) from {username='oryx', principal='oryx/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:36580
I20260808 07:02:05.671526 21680 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c55b358933944eeb406288477bc84ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:05 UTC) from {username='oryx', principal='oryx/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:51936
I20260808 07:02:05.672071 21808 tablet_replica.cc:330] T 6c55b358933944eeb406288477bc84ff P acb5dce36f6043409bc52171d9ba692f: stopping tablet replica
I20260808 07:02:05.672236 21808 raft_consensus.cc:2229] T 6c55b358933944eeb406288477bc84ff P acb5dce36f6043409bc52171d9ba692f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:02:05.672307 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 20427
I20260808 07:02:05.673216 21808 raft_consensus.cc:2258] T 6c55b358933944eeb406288477bc84ff P acb5dce36f6043409bc52171d9ba692f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:05.673880 21808 ts_tablet_manager.cc:1905] T 6c55b358933944eeb406288477bc84ff P acb5dce36f6043409bc52171d9ba692f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:05.676349 21808 ts_tablet_manager.cc:1918] T 6c55b358933944eeb406288477bc84ff P acb5dce36f6043409bc52171d9ba692f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:02:05.676481 21808 log.cc:1196] T 6c55b358933944eeb406288477bc84ff P acb5dce36f6043409bc52171d9ba692f: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/6c55b358933944eeb406288477bc84ff
I20260808 07:02:05.676887 21808 ts_tablet_manager.cc:1939] T 6c55b358933944eeb406288477bc84ff P acb5dce36f6043409bc52171d9ba692f: Deleting consensus metadata
I20260808 07:02:05.678922 21805 tablet_replica.cc:330] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86: stopping tablet replica
I20260808 07:02:05.679107 21805 raft_consensus.cc:2229] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:02:05.679382 21805 raft_consensus.cc:2258] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:05.680080 21805 ts_tablet_manager.cc:1905] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:05.682823 21807 tablet_replica.cc:330] T 6c55b358933944eeb406288477bc84ff P 76046ba280e44aba95b3074f9a8701d1: stopping tablet replica
I20260808 07:02:05.683007 21807 raft_consensus.cc:2229] T 6c55b358933944eeb406288477bc84ff P 76046ba280e44aba95b3074f9a8701d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:02:05.683152 21807 raft_consensus.cc:2258] T 6c55b358933944eeb406288477bc84ff P 76046ba280e44aba95b3074f9a8701d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:05.683200 21805 ts_tablet_manager.cc:1918] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:02:05.683310 21805 log.cc:1196] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/wal/wals/6c55b358933944eeb406288477bc84ff
W20260808 07:02:05.683404 21631 connection.cc:535] server connection from 127.0.0.1:51936 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 07:02:05.683820 21805 ts_tablet_manager.cc:1939] T 6c55b358933944eeb406288477bc84ff P 5426123c2e6b448d89af9dbe4b4c9a86: Deleting consensus metadata
I20260808 07:02:05.684152 21807 ts_tablet_manager.cc:1905] T 6c55b358933944eeb406288477bc84ff P 76046ba280e44aba95b3074f9a8701d1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 07:02:05.684336 21489 connection.cc:441] server connection from 127.0.0.1:40658 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40658 (request call id 1) could send its response
I20260808 07:02:05.688138 21807 ts_tablet_manager.cc:1918] T 6c55b358933944eeb406288477bc84ff P 76046ba280e44aba95b3074f9a8701d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:02:05.688266 21807 log.cc:1196] T 6c55b358933944eeb406288477bc84ff P 76046ba280e44aba95b3074f9a8701d1: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-0/wal/wals/6c55b358933944eeb406288477bc84ff
I20260808 07:02:05.688625 21807 ts_tablet_manager.cc:1939] T 6c55b358933944eeb406288477bc84ff P 76046ba280e44aba95b3074f9a8701d1: Deleting consensus metadata
W20260808 07:02:05.689277 21343 connection.cc:441] server connection from 127.0.0.1:36580 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36580 (request call id 1) could send its response
I20260808 07:02:05.693166 21764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.671310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36822 (local address 127.13.207.60:32895)
0808 07:02:05.671397 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:05.671400 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:05.683458 (+ 12058us) server_negotiation.cc:407] Connection header received
0808 07:02:05.683593 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:05.683595 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:05.683616 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:05.683652 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:05.684744 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.685647 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.686782 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.687116 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.687166 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.689740 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:05.689742 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:05.689742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:05.689752 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:05.691726 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.692182 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.692184 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.692184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.692244 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.692441 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.692442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.692442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.692577 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:05.692671 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:05.692960 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 07:02:05.693058 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 07:02:05.696913 21761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.686828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36510 (local address 127.13.207.61:38765)
0808 07:02:05.686885 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:05.686887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:05.687588 (+ 701us) server_negotiation.cc:407] Connection header received
0808 07:02:05.687694 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:05.687695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:05.687713 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:05.687747 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:05.689091 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.689918 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.691150 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.691488 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.691536 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.693113 (+ 1577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:05.693116 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:05.693116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:05.693125 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:05.695538 (+ 2413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.695986 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.695988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.695988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.696055 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.696278 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.696279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.696280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.696416 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:05.696540 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:05.696678 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 07:02:05.696791 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 07:02:05.697530 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:05.687191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36828 (local address 127.13.207.60:32895)
0808 07:02:05.687235 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:05.687237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:05.688189 (+ 952us) server_negotiation.cc:407] Connection header received
0808 07:02:05.689802 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:05.689804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:05.689815 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:05.689844 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:05.690532 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.691261 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.692312 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:05.692648 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:05.692697 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:05.694327 (+ 1630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:05.694328 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:05.694328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:05.694336 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:05.696139 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.696585 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.696586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.696587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.696653 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:05.696877 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:05.696878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:05.696879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:05.697027 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:05.697141 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:05.697341 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 07:02:05.697433 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:02:06.664631 21816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:06.664239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.13.207.3:47371)
0808 07:02:06.664391 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:06.664570 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
W20260808 07:02:06.664893 21749 heartbeater.cc:644] Failed to heartbeat to 127.13.207.62:35437 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
I20260808 07:02:06.665081 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:06.664679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.13.207.1:43087)
0808 07:02:06.664814 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:06.665034 (+ 220us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
W20260808 07:02:06.665343 21462 heartbeater.cc:644] Failed to heartbeat to 127.13.207.62:35437 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
I20260808 07:02:06.665611 21818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:06.665154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.13.207.2:60507)
0808 07:02:06.665491 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:06.665565 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":57,"threads_started":1}
W20260808 07:02:06.665833 21606 heartbeater.cc:644] Failed to heartbeat to 127.13.207.62:35437 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
I20260808 07:02:07.169869 21819 raft_consensus.cc:482] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 109a60c497f74cb5bbb02a560ae83bc1)
I20260808 07:02:07.170097 21819 raft_consensus.cc:504] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: false } }
I20260808 07:02:07.170385 21819 leader_election.cc:288] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 109a60c497f74cb5bbb02a560ae83bc1 (127.13.207.62:35437), 92e4477c73b24415b051967647dac505 (127.13.207.61:38765)
I20260808 07:02:07.170959 21822 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 109a60c497f74cb5bbb02a560ae83bc1)
I20260808 07:02:07.171073 21821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.170761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.0.0.1:38238)
0808 07:02:07.170960 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.171019 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":152,"threads_started":1}
W20260808 07:02:07.171192 21260 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
I20260808 07:02:07.171351 21822 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: false } }
I20260808 07:02:07.171674 21822 leader_election.cc:288] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 109a60c497f74cb5bbb02a560ae83bc1 (127.13.207.62:35437), e93d3d4db1114d719e7c2d6f99456785 (127.13.207.60:32895)
I20260808 07:02:07.171998 21826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.171772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.0.0.1:38256)
0808 07:02:07.171893 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.171958 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260808 07:02:07.172111 21823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.171514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.0.0.1:38242)
0808 07:02:07.172044 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.172078 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510}
W20260808 07:02:07.172204 21066 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
W20260808 07:02:07.172291 21260 leader_election.cc:334] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 109a60c497f74cb5bbb02a560ae83bc1 (127.13.207.62:35437): Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
I20260808 07:02:07.172761 21826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.172586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.0.0.1:38260)
0808 07:02:07.172700 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.172731 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":82,"threads_started":1}
W20260808 07:02:07.172892 21066 leader_election.cc:334] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 109a60c497f74cb5bbb02a560ae83bc1 (127.13.207.62:35437): Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
I20260808 07:02:07.175113 21821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.171048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:38765 (local address 127.0.0.1:36526)
0808 07:02:07.171139 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.171149 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:07.171210 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:07.171417 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:07.171418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:07.171422 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:07.171488 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:07.171490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.172538 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.172539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:07.173962 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:07.173965 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.174847 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.174848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:07.174902 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:07.174908 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:02:07.175008 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 07:02:07.175048 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":63,"threads_started":1}
I20260808 07:02:07.175145 21824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.171146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36526 (local address 127.13.207.61:38765)
0808 07:02:07.171243 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:07.171245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:07.171256 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:02:07.171289 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:07.171290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:07.171302 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:07.171398 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:07.171605 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.172406 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.174052 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.174726 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.174826 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:07.174841 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:07.175030 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 07:02:07.175075 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":46,"threads_started":1}
I20260808 07:02:07.175508 21090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e93d3d4db1114d719e7c2d6f99456785" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "92e4477c73b24415b051967647dac505" is_pre_election: true
I20260808 07:02:07.175638 21090 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e93d3d4db1114d719e7c2d6f99456785 in term 1.
I20260808 07:02:07.175813 21260 leader_election.cc:302] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92e4477c73b24415b051967647dac505, e93d3d4db1114d719e7c2d6f99456785; no voters: 109a60c497f74cb5bbb02a560ae83bc1
I20260808 07:02:07.175968 21819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 07:02:07.176039 21819 raft_consensus.cc:482] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 109a60c497f74cb5bbb02a560ae83bc1)
I20260808 07:02:07.176071 21819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:02:07.176436 21828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.172165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36830 (local address 127.13.207.60:32895)
0808 07:02:07.172375 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:07.172377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:07.172383 (+ 6us) server_negotiation.cc:407] Connection header received
0808 07:02:07.172405 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:07.172406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:07.172417 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:07.172510 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:07.172803 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.173677 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.175428 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.176056 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.176148 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:07.176160 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:07.176323 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 07:02:07.176370 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":97,"server-negotiator.queue_time_us":180,"thread_start_us":89,"threads_started":1}
I20260808 07:02:07.176462 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.171950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:32895 (local address 127.0.0.1:36830)
0808 07:02:07.172207 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.172217 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:07.172277 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:07.172524 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:07.172525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:07.172528 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:07.172582 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:07.172584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.173862 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.173863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:07.175239 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:07.175241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.176150 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.176151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:07.176221 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:07.176225 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:02:07.176314 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 07:02:07.176380 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":40,"thread_start_us":40,"threads_started":1}
I20260808 07:02:07.176769 21285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92e4477c73b24415b051967647dac505" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e93d3d4db1114d719e7c2d6f99456785" is_pre_election: true
I20260808 07:02:07.177031 21819 raft_consensus.cc:504] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: false } }
I20260808 07:02:07.177189 21819 leader_election.cc:288] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [CANDIDATE]: Term 2 election: Requested vote from peers 109a60c497f74cb5bbb02a560ae83bc1 (127.13.207.62:35437), 92e4477c73b24415b051967647dac505 (127.13.207.61:38765)
I20260808 07:02:07.177268 21285 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92e4477c73b24415b051967647dac505 in current term 2: Already voted for candidate e93d3d4db1114d719e7c2d6f99456785 in this term.
I20260808 07:02:07.177523 21821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.177401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.0.0.1:38270)
0808 07:02:07.177454 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.177494 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 07:02:07.177575 21066 leader_election.cc:302] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92e4477c73b24415b051967647dac505; no voters: 109a60c497f74cb5bbb02a560ae83bc1, e93d3d4db1114d719e7c2d6f99456785
I20260808 07:02:07.177745 21090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e93d3d4db1114d719e7c2d6f99456785" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "92e4477c73b24415b051967647dac505"
I20260808 07:02:07.177845 21090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:02:07.178050 21821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.177906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.0.0.1:38282)
0808 07:02:07.177983 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.178022 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260808 07:02:07.178268 21260 leader_election.cc:334] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 109a60c497f74cb5bbb02a560ae83bc1 (127.13.207.62:35437): Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
I20260808 07:02:07.178575 21090 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e93d3d4db1114d719e7c2d6f99456785 in term 2.
I20260808 07:02:07.178668 21822 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:02:07.178835 21260 leader_election.cc:302] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92e4477c73b24415b051967647dac505, e93d3d4db1114d719e7c2d6f99456785; no voters: 109a60c497f74cb5bbb02a560ae83bc1
I20260808 07:02:07.178967 21819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 07:02:07.179203 21819 raft_consensus.cc:686] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [term 2 LEADER]: Becoming Leader. State: Replica: e93d3d4db1114d719e7c2d6f99456785, State: Running, Role: LEADER
I20260808 07:02:07.179307 21819 consensus_queue.cc:237] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: false } }
I20260808 07:02:07.179845 21832 sys_catalog.cc:455] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e93d3d4db1114d719e7c2d6f99456785. Latest consensus state: current_term: 2 leader_uuid: "e93d3d4db1114d719e7c2d6f99456785" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: false } } }
I20260808 07:02:07.179955 21832 sys_catalog.cc:458] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:07.180155 21834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:02:07.180576 21834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:02:07.180636 21834 catalog_manager.cc:1259] Loaded cluster ID: 8804463ead1f41d69e60b49669459c0f
I20260808 07:02:07.180653 21834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:02:07.180837 21834 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:02:07.180914 21834 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785: Loaded TSK: 0
I20260808 07:02:07.181079 21834 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:02:07.202400 21274 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36828:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:02:07.204358 21274 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:02:07.205751 21821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.205623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.0.0.1:38298)
0808 07:02:07.205682 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.205721 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 07:02:07.205775 21090 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [term 2 FOLLOWER]: Refusing update from remote peer e93d3d4db1114d719e7c2d6f99456785: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 07:02:07.206336 21821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.206120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.0.0.1:38312)
0808 07:02:07.206200 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.206271 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 07:02:07.206270 21832 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260808 07:02:07.206548 21260 consensus_peers.cc:487] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 -> Peer 109a60c497f74cb5bbb02a560ae83bc1 (127.13.207.62:35437): Couldn't send request to peer 109a60c497f74cb5bbb02a560ae83bc1. Status: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 07:02:07.207193 21822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e93d3d4db1114d719e7c2d6f99456785. Latest consensus state: current_term: 2 leader_uuid: "e93d3d4db1114d719e7c2d6f99456785" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: false } } }
I20260808 07:02:07.207301 21822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:02:07.207938 21832 sys_catalog.cc:455] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "e93d3d4db1114d719e7c2d6f99456785" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: false } } }
I20260808 07:02:07.208046 21832 sys_catalog.cc:458] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:07.208271 21822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "e93d3d4db1114d719e7c2d6f99456785" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "109a60c497f74cb5bbb02a560ae83bc1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35437 } } peers { permanent_uuid: "92e4477c73b24415b051967647dac505" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 38765 } attrs { promote: false } } peers { permanent_uuid: "e93d3d4db1114d719e7c2d6f99456785" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 32895 } attrs { promote: false } } }
I20260808 07:02:07.208380 21822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92e4477c73b24415b051967647dac505 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:02:07.208629 21837 mvcc.cc:205] Tried to move back new op lower bound from 7316162671432032256 to 7316162671327686656. Current Snapshot: MvccSnapshot[applied={T|T < 7316162671432032256}]
I20260808 07:02:07.213738 21823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.210024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:46325 (local address 127.0.0.1:51942)
0808 07:02:07.210056 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.210066 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:07.210114 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:07.210635 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:07.210635 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:07.210639 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:07.210698 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:07.210700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.211849 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.211850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:07.212748 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:07.212751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.213470 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.213471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:07.213561 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:07.213566 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:02:07.213599 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 07:02:07.213645 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":34}
I20260808 07:02:07.214102 21844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.210341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51942 (local address 127.13.207.3:46325)
0808 07:02:07.210483 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:07.210485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:07.210493 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:02:07.210518 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:07.210519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:07.210529 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:07.210762 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:07.210802 (+ 40us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.211740 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.212849 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.213356 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.213916 (+ 560us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:07.213933 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:07.213957 (+ 24us) server_negotiation.cc:299] Negotiation successful
0808 07:02:07.214006 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
I20260808 07:02:07.214258 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.210121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:45561 (local address 127.0.0.1:36590)
0808 07:02:07.210255 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.210266 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:07.210330 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:07.210705 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:07.210706 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:07.210708 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:07.210771 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:07.210772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.211451 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.211452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:07.212790 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:07.212793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.213674 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.213675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:07.213747 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:07.213751 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:02:07.214114 (+ 363us) client_negotiation.cc:240] Negotiation successful
0808 07:02:07.214164 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260808 07:02:07.214370 21821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.209970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:38057 (local address 127.0.0.1:40690)
0808 07:02:07.210006 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.210018 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:07.210071 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:07.210463 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:07.210464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:07.210469 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:07.210533 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:07.210535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.211783 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.211784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:07.213140 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:07.213142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.214109 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.214110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:07.214169 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:07.214172 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:02:07.214224 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:02:07.214259 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260808 07:02:07.214341 21680 tablet_service.cc:1467] Processing CreateTablet for tablet 58026fea15104d6db9ba6c6914d324c0 (DEFAULT_TABLE table=test-table [id=58c6c0476125484d8e06ab25f9c6ff0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:07.214530 21680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58026fea15104d6db9ba6c6914d324c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:07.214535 21845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.210391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36590 (local address 127.13.207.1:45561)
0808 07:02:07.210521 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:07.210523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:07.210531 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:02:07.210557 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:07.210557 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:07.210567 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:07.210630 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:07.210869 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.211357 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.212961 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.213578 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.213663 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:07.213681 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:07.214380 (+ 699us) server_negotiation.cc:299] Negotiation successful
0808 07:02:07.214441 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260808 07:02:07.214773 21842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.210056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40690 (local address 127.13.207.2:38057)
0808 07:02:07.210203 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:07.210206 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:07.210213 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:02:07.210246 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:07.210247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:07.210256 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:07.210610 (+ 354us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:07.210656 (+ 46us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.211679 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.213239 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.214027 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.214097 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:07.214110 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:07.214296 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 07:02:07.214575 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260808 07:02:07.214772 21393 tablet_service.cc:1467] Processing CreateTablet for tablet 58026fea15104d6db9ba6c6914d324c0 (DEFAULT_TABLE table=test-table [id=58c6c0476125484d8e06ab25f9c6ff0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:07.214912 21393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58026fea15104d6db9ba6c6914d324c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:07.216075 21846 tablet_bootstrap.cc:492] T 58026fea15104d6db9ba6c6914d324c0 P acb5dce36f6043409bc52171d9ba692f: Bootstrap starting.
I20260808 07:02:07.216248 21847 tablet_bootstrap.cc:492] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1: Bootstrap starting.
I20260808 07:02:07.216228 21537 tablet_service.cc:1467] Processing CreateTablet for tablet 58026fea15104d6db9ba6c6914d324c0 (DEFAULT_TABLE table=test-table [id=58c6c0476125484d8e06ab25f9c6ff0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:07.216356 21537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58026fea15104d6db9ba6c6914d324c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:07.216609 21846 tablet_bootstrap.cc:654] T 58026fea15104d6db9ba6c6914d324c0 P acb5dce36f6043409bc52171d9ba692f: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:07.216699 21847 tablet_bootstrap.cc:654] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:07.217258 21846 tablet_bootstrap.cc:492] T 58026fea15104d6db9ba6c6914d324c0 P acb5dce36f6043409bc52171d9ba692f: No bootstrap required, opened a new log
I20260808 07:02:07.217331 21846 ts_tablet_manager.cc:1397] T 58026fea15104d6db9ba6c6914d324c0 P acb5dce36f6043409bc52171d9ba692f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:07.217463 21846 raft_consensus.cc:348] T 58026fea15104d6db9ba6c6914d324c0 P acb5dce36f6043409bc52171d9ba692f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38057 } } peers { permanent_uuid: "acb5dce36f6043409bc52171d9ba692f" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46325 } } peers { permanent_uuid: "76046ba280e44aba95b3074f9a8701d1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45561 } }
I20260808 07:02:07.217554 21846 raft_consensus.cc:374] T 58026fea15104d6db9ba6c6914d324c0 P acb5dce36f6043409bc52171d9ba692f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:07.217579 21846 raft_consensus.cc:729] T 58026fea15104d6db9ba6c6914d324c0 P acb5dce36f6043409bc52171d9ba692f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb5dce36f6043409bc52171d9ba692f, State: Initialized, Role: FOLLOWER
I20260808 07:02:07.217635 21846 consensus_queue.cc:260] T 58026fea15104d6db9ba6c6914d324c0 P acb5dce36f6043409bc52171d9ba692f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38057 } } peers { permanent_uuid: "acb5dce36f6043409bc52171d9ba692f" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46325 } } peers { permanent_uuid: "76046ba280e44aba95b3074f9a8701d1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45561 } }
I20260808 07:02:07.217811 21846 ts_tablet_manager.cc:1428] T 58026fea15104d6db9ba6c6914d324c0 P acb5dce36f6043409bc52171d9ba692f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:02:07.217842 21851 tablet_bootstrap.cc:492] T 58026fea15104d6db9ba6c6914d324c0 P 5426123c2e6b448d89af9dbe4b4c9a86: Bootstrap starting.
I20260808 07:02:07.218186 21847 tablet_bootstrap.cc:492] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1: No bootstrap required, opened a new log
I20260808 07:02:07.218245 21847 ts_tablet_manager.cc:1397] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:02:07.218336 21851 tablet_bootstrap.cc:654] T 58026fea15104d6db9ba6c6914d324c0 P 5426123c2e6b448d89af9dbe4b4c9a86: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:07.218362 21852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.218109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.13.207.3:52553)
0808 07:02:07.218260 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.218313 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
I20260808 07:02:07.218422 21847 raft_consensus.cc:348] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38057 } } peers { permanent_uuid: "acb5dce36f6043409bc52171d9ba692f" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46325 } } peers { permanent_uuid: "76046ba280e44aba95b3074f9a8701d1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45561 } }
I20260808 07:02:07.218506 21847 raft_consensus.cc:374] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:07.218528 21847 raft_consensus.cc:729] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76046ba280e44aba95b3074f9a8701d1, State: Initialized, Role: FOLLOWER
I20260808 07:02:07.218573 21847 consensus_queue.cc:260] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38057 } } peers { permanent_uuid: "acb5dce36f6043409bc52171d9ba692f" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46325 } } peers { permanent_uuid: "76046ba280e44aba95b3074f9a8701d1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45561 } }
I20260808 07:02:07.218765 21847 ts_tablet_manager.cc:1428] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:02:07.218978 21851 tablet_bootstrap.cc:492] T 58026fea15104d6db9ba6c6914d324c0 P 5426123c2e6b448d89af9dbe4b4c9a86: No bootstrap required, opened a new log
I20260808 07:02:07.219033 21851 ts_tablet_manager.cc:1397] T 58026fea15104d6db9ba6c6914d324c0 P 5426123c2e6b448d89af9dbe4b4c9a86: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:07.219374 21748 heartbeater.cc:503] Master 127.13.207.60:32895 was elected leader, sending a full tablet report...
I20260808 07:02:07.219303 21851 raft_consensus.cc:348] T 58026fea15104d6db9ba6c6914d324c0 P 5426123c2e6b448d89af9dbe4b4c9a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38057 } } peers { permanent_uuid: "acb5dce36f6043409bc52171d9ba692f" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46325 } } peers { permanent_uuid: "76046ba280e44aba95b3074f9a8701d1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45561 } }
I20260808 07:02:07.219509 21851 raft_consensus.cc:374] T 58026fea15104d6db9ba6c6914d324c0 P 5426123c2e6b448d89af9dbe4b4c9a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:07.219550 21851 raft_consensus.cc:729] T 58026fea15104d6db9ba6c6914d324c0 P 5426123c2e6b448d89af9dbe4b4c9a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5426123c2e6b448d89af9dbe4b4c9a86, State: Initialized, Role: FOLLOWER
I20260808 07:02:07.219602 21851 consensus_queue.cc:260] T 58026fea15104d6db9ba6c6914d324c0 P 5426123c2e6b448d89af9dbe4b4c9a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38057 } } peers { permanent_uuid: "acb5dce36f6043409bc52171d9ba692f" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46325 } } peers { permanent_uuid: "76046ba280e44aba95b3074f9a8701d1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45561 } }
I20260808 07:02:07.219758 21851 ts_tablet_manager.cc:1428] T 58026fea15104d6db9ba6c6914d324c0 P 5426123c2e6b448d89af9dbe4b4c9a86: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:07.219929 21461 heartbeater.cc:503] Master 127.13.207.60:32895 was elected leader, sending a full tablet report...
I20260808 07:02:07.220466 21605 heartbeater.cc:503] Master 127.13.207.60:32895 was elected leader, sending a full tablet report...
I20260808 07:02:07.220498 21855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.219139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.13.207.1:60775)
0808 07:02:07.220097 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.220434 (+ 337us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":925,"thread_start_us":89,"threads_started":1}
I20260808 07:02:07.220947 21852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.220803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.13.207.3:41315)
0808 07:02:07.220873 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.220912 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 07:02:07.221026 21855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.220711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.13.207.1:52881)
0808 07:02:07.220970 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.220998 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":82,"threads_started":1}
I20260808 07:02:07.221257 21858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.219990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.13.207.2:35325)
0808 07:02:07.221187 (+ 1197us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.221225 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1171,"thread_start_us":87,"threads_started":1}
I20260808 07:02:07.221616 21858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.221527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.13.207.2:32999)
0808 07:02:07.221575 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.221599 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 07:02:07.343861 21854 raft_consensus.cc:482] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:02:07.344069 21854 raft_consensus.cc:504] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38057 } } peers { permanent_uuid: "acb5dce36f6043409bc52171d9ba692f" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46325 } } peers { permanent_uuid: "76046ba280e44aba95b3074f9a8701d1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45561 } }
I20260808 07:02:07.344632 21854 leader_election.cc:288] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5426123c2e6b448d89af9dbe4b4c9a86 (127.13.207.2:38057), acb5dce36f6043409bc52171d9ba692f (127.13.207.3:46325)
I20260808 07:02:07.349020 21855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.344889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:46325 (local address 127.13.207.1:36349)
0808 07:02:07.344991 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.345004 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:07.345069 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:07.345335 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:07.345337 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:07.345338 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:07.345458 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:07.345461 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.346406 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.346408 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:07.347840 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:07.347844 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.348680 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.348682 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:07.348775 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:07.348781 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:02:07.348862 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 07:02:07.348915 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":56}
I20260808 07:02:07.349020 21844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.345090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:36349 (local address 127.13.207.3:46325)
0808 07:02:07.345147 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:07.345148 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:07.345156 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:02:07.345183 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:07.345184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:07.345192 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:07.345299 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:07.345544 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.346304 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.347952 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.348548 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.348638 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:07.348649 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:07.348883 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 07:02:07.348929 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:02:07.349098 21842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.345021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:46563 (local address 127.13.207.2:38057)
0808 07:02:07.345071 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:07.345073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:07.345228 (+ 155us) server_negotiation.cc:407] Connection header received
0808 07:02:07.345258 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:07.345258 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:07.345269 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:07.345327 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:07.345545 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.346298 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.347952 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.348547 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.348638 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:07.348649 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:07.348986 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 07:02:07.349035 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:02:07.349308 21859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.344889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:38057 (local address 127.13.207.1:46563)
0808 07:02:07.345150 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.345160 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:07.345208 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:07.345341 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:07.345342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:07.345343 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:07.345458 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:07.345461 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.346406 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.346408 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:07.347840 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:07.347844 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.348707 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.348708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:07.348775 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:07.348781 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:02:07.348962 (+ 181us) client_negotiation.cc:240] Negotiation successful
0808 07:02:07.349001 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":140}
I20260808 07:02:07.349313 21701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58026fea15104d6db9ba6c6914d324c0" candidate_uuid: "76046ba280e44aba95b3074f9a8701d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb5dce36f6043409bc52171d9ba692f" is_pre_election: true
I20260808 07:02:07.349498 21701 raft_consensus.cc:2454] T 58026fea15104d6db9ba6c6914d324c0 P acb5dce36f6043409bc52171d9ba692f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76046ba280e44aba95b3074f9a8701d1 in term 0.
I20260808 07:02:07.349545 21558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58026fea15104d6db9ba6c6914d324c0" candidate_uuid: "76046ba280e44aba95b3074f9a8701d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" is_pre_election: true
I20260808 07:02:07.349721 21558 raft_consensus.cc:2454] T 58026fea15104d6db9ba6c6914d324c0 P 5426123c2e6b448d89af9dbe4b4c9a86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76046ba280e44aba95b3074f9a8701d1 in term 0.
I20260808 07:02:07.349767 21346 leader_election.cc:302] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76046ba280e44aba95b3074f9a8701d1, acb5dce36f6043409bc52171d9ba692f; no voters:
I20260808 07:02:07.350001 21854 raft_consensus.cc:2781] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:02:07.350051 21854 raft_consensus.cc:482] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:02:07.350068 21854 raft_consensus.cc:3037] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:07.350908 21854 raft_consensus.cc:504] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38057 } } peers { permanent_uuid: "acb5dce36f6043409bc52171d9ba692f" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46325 } } peers { permanent_uuid: "76046ba280e44aba95b3074f9a8701d1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45561 } }
I20260808 07:02:07.351080 21854 leader_election.cc:288] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1 [CANDIDATE]: Term 1 election: Requested vote from peers 5426123c2e6b448d89af9dbe4b4c9a86 (127.13.207.2:38057), acb5dce36f6043409bc52171d9ba692f (127.13.207.3:46325)
I20260808 07:02:07.351238 21558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58026fea15104d6db9ba6c6914d324c0" candidate_uuid: "76046ba280e44aba95b3074f9a8701d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5426123c2e6b448d89af9dbe4b4c9a86"
I20260808 07:02:07.351269 21701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58026fea15104d6db9ba6c6914d324c0" candidate_uuid: "76046ba280e44aba95b3074f9a8701d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb5dce36f6043409bc52171d9ba692f"
I20260808 07:02:07.351332 21558 raft_consensus.cc:3037] T 58026fea15104d6db9ba6c6914d324c0 P 5426123c2e6b448d89af9dbe4b4c9a86 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:07.351342 21701 raft_consensus.cc:3037] T 58026fea15104d6db9ba6c6914d324c0 P acb5dce36f6043409bc52171d9ba692f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:07.352231 21701 raft_consensus.cc:2454] T 58026fea15104d6db9ba6c6914d324c0 P acb5dce36f6043409bc52171d9ba692f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76046ba280e44aba95b3074f9a8701d1 in term 1.
I20260808 07:02:07.352277 21558 raft_consensus.cc:2454] T 58026fea15104d6db9ba6c6914d324c0 P 5426123c2e6b448d89af9dbe4b4c9a86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76046ba280e44aba95b3074f9a8701d1 in term 1.
I20260808 07:02:07.352474 21344 leader_election.cc:302] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5426123c2e6b448d89af9dbe4b4c9a86, 76046ba280e44aba95b3074f9a8701d1; no voters:
I20260808 07:02:07.352623 21854 raft_consensus.cc:2781] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:02:07.352825 21854 raft_consensus.cc:686] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1 [term 1 LEADER]: Becoming Leader. State: Replica: 76046ba280e44aba95b3074f9a8701d1, State: Running, Role: LEADER
I20260808 07:02:07.352924 21854 consensus_queue.cc:237] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38057 } } peers { permanent_uuid: "acb5dce36f6043409bc52171d9ba692f" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46325 } } peers { permanent_uuid: "76046ba280e44aba95b3074f9a8701d1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45561 } }
I20260808 07:02:07.353302 21859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.353176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.13.207.1:55969)
0808 07:02:07.353226 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.353262 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 07:02:07.353855 21274 catalog_manager.cc:5495] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76046ba280e44aba95b3074f9a8701d1 (127.13.207.1). New cstate: current_term: 1 leader_uuid: "76046ba280e44aba95b3074f9a8701d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acb5dce36f6043409bc52171d9ba692f" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76046ba280e44aba95b3074f9a8701d1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45561 } health_report { overall_health: HEALTHY } } }
I20260808 07:02:07.365096 21845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.362328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36604 (local address 127.13.207.1:45561)
0808 07:02:07.362399 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:07.362402 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:07.362414 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:02:07.362505 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:07.362506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:07.362525 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:07.362614 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:02:07.362774 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.363272 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.364104 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:07.364309 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:07.364395 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:07.364575 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:02:07.364684 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:02:07.364767 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:07.364821 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 07:02:07.364847 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:02:07.366441 21701 raft_consensus.cc:1261] T 58026fea15104d6db9ba6c6914d324c0 P acb5dce36f6043409bc52171d9ba692f [term 1 FOLLOWER]: Refusing update from remote peer 76046ba280e44aba95b3074f9a8701d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:07.366602 21558 raft_consensus.cc:1261] T 58026fea15104d6db9ba6c6914d324c0 P 5426123c2e6b448d89af9dbe4b4c9a86 [term 1 FOLLOWER]: Refusing update from remote peer 76046ba280e44aba95b3074f9a8701d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:07.366814 21860 consensus_queue.cc:1035] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb5dce36f6043409bc52171d9ba692f" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 46325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:07.366951 21854 consensus_queue.cc:1035] T 58026fea15104d6db9ba6c6914d324c0 P 76046ba280e44aba95b3074f9a8701d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5426123c2e6b448d89af9dbe4b4c9a86" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:07.368317 21852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.367962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.13.207.3:40977)
0808 07:02:07.368218 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.368270 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197}
I20260808 07:02:07.368541 21858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.368390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.13.207.2:42851)
0808 07:02:07.368478 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.368506 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 07:02:07.369736 21858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.369632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.13.207.2:36117)
0808 07:02:07.369668 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.369701 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260808 07:02:07.369807 21859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.369638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.13.207.1:57241)
0808 07:02:07.369741 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.369771 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 07:02:07.370177 21865 mvcc.cc:205] Tried to move back new op lower bound from 7316162672090550272 to 7316162672038473728. Current Snapshot: MvccSnapshot[applied={T|T < 7316162672090550272}]
I20260808 07:02:07.371308 21859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.371195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.13.207.1:59735)
0808 07:02:07.371235 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.371271 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 07:02:07.372020 21852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.371652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35437 (local address 127.13.207.3:54259)
0808 07:02:07.371944 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.371978 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262}
I20260808 07:02:07.372583 21274 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:02:07.372694 21274 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:02:07.374275 21274 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e93d3d4db1114d719e7c2d6f99456785: Sending DeleteTablet for 3 replicas of tablet 58026fea15104d6db9ba6c6914d324c0
I20260808 07:02:07.374639 21537 tablet_service.cc:1514] Processing DeleteTablet for tablet 58026fea15104d6db9ba6c6914d324c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:07 UTC) from {username='oryx', principal='oryx/127.13.207.60@KRBTEST.COM'} at 127.0.0.1:40690
I20260808 07:02:07.374955 21873 tablet_replica.cc:330] T 58026fea15104d6db9ba6c6914d324c0 P 5426123c2e6b448d89af9dbe4b4c9a86: stopping tablet replica
I20260808 07:02:07.374969 21680 tablet_service.cc:1514] Processing DeleteTablet for tablet 58026fea15104d6db9ba6c6914d324c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:07 UTC) from {username='oryx', principal='oryx/127.13.207.60@KRBTEST.COM'} at 127.0.0.1:51942
I20260808 07:02:07.375042 21393 tablet_service.cc:1514] Processing DeleteTablet for tablet 58026fea15104d6db9ba6c6914d324c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:07 UTC) from {username='oryx', principal='oryx/127.13.207.60@KRBTEST.COM'} at 127.0.0.1:36590
I20260808 07:02:07.375162 21874 tablet_replica.cc:330] T 58026fea15104d6db9ba6c6914d324c0 P acb5dce36f6043409bc52171d9ba692f: stopping tablet replica
I20260808 07:02:07.375226 21874 raft_consensus.cc:2229] T 58026fea15104d6db9ba6c6914d324c0 P acb5dce36f6043409bc52171d9ba692f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:02:07.375175 21873 raft_consensus.cc:2229] T 58026fea15104d6db9ba6c6914d324c0 P 5426123c2e6b448d89af9dbe4b4c9a86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:02:07.375284 21874 raft_consensus.cc:2258] T 58026fea15104d6db9ba6c6914d324c0 P acb5dce36f6043409bc52171d9ba692f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:07.375380 21873 raft_consensus.cc:2258] T 58026fea15104d6db9ba6c6914d324c0 P 5426123c2e6b448d89af9dbe4b4c9a86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:07.375706 21874 ts_tablet_manager.cc:1905] T 58026fea15104d6db9ba6c6914d324c0 P acb5dce36f6043409bc52171d9ba692f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:07.375766 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 21231
I20260808 07:02:07.375754 21873 ts_tablet_manager.cc:1905] T 58026fea15104d6db9ba6c6914d324c0 P 5426123c2e6b448d89af9dbe4b4c9a86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:07.377650 21874 ts_tablet_manager.cc:1918] T 58026fea15104d6db9ba6c6914d324c0 P acb5dce36f6043409bc52171d9ba692f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:02:07.377712 21874 log.cc:1196] T 58026fea15104d6db9ba6c6914d324c0 P acb5dce36f6043409bc52171d9ba692f: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/58026fea15104d6db9ba6c6914d324c0
I20260808 07:02:07.378074 21874 ts_tablet_manager.cc:1939] T 58026fea15104d6db9ba6c6914d324c0 P acb5dce36f6043409bc52171d9ba692f: Deleting consensus metadata
I20260808 07:02:07.378077 21873 ts_tablet_manager.cc:1918] T 58026fea15104d6db9ba6c6914d324c0 P 5426123c2e6b448d89af9dbe4b4c9a86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:02:07.378196 21873 log.cc:1196] T 58026fea15104d6db9ba6c6914d324c0 P 5426123c2e6b448d89af9dbe4b4c9a86: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172493058068-14140-0/minicluster-data/ts-1/wal/wals/58026fea15104d6db9ba6c6914d324c0
I20260808 07:02:07.378479 21873 ts_tablet_manager.cc:1939] T 58026fea15104d6db9ba6c6914d324c0 P 5426123c2e6b448d89af9dbe4b4c9a86: Deleting consensus metadata
I20260808 07:02:07.378453 21260 catalog_manager.cc:4841] TS acb5dce36f6043409bc52171d9ba692f (127.13.207.3:46325): tablet 58026fea15104d6db9ba6c6914d324c0 (table test-table [id=58c6c0476125484d8e06ab25f9c6ff0a]) successfully deleted
I20260808 07:02:07.378978 21259 catalog_manager.cc:4841] TS 5426123c2e6b448d89af9dbe4b4c9a86 (127.13.207.2:38057): tablet 58026fea15104d6db9ba6c6914d324c0 (table test-table [id=58c6c0476125484d8e06ab25f9c6ff0a]) successfully deleted
I20260808 07:02:07.384120 21821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:07.383967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:45561 (local address 127.0.0.1:36610)
0808 07:02:07.384045 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:07.384081 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.1:45561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 07:02:07.384277 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 21471
W20260808 07:02:07.384382 21260 catalog_manager.cc:4584] TS 76046ba280e44aba95b3074f9a8701d1 (127.13.207.1:45561): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 58026fea15104d6db9ba6c6914d324c0: Network error: Client connection negotiation failed: client connection to 127.13.207.1:45561: connect: Connection refused (error 111)
I20260808 07:02:07.392216 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 21612
I20260808 07:02:07.400063 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 20942
I20260808 07:02:07.407011 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 21135
2026-08-08T07:02:07Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5305 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:02:07 dist-test-slave-8tdl krb5kdc[21878](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:02:07 dist-test-slave-8tdl krb5kdc[21878](info): set up 2 sockets
Aug 08 07:02:07 dist-test-slave-8tdl krb5kdc[21878](info): commencing operation
krb5kdc: starting...
W20260808 07:02:08.555794 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:02:08 dist-test-slave-8tdl krb5kdc[21878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:02:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:02:08Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.62.keytab.
Entry for principal oryx/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.62.keytab.
I20260808 07:02:08.715405 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:38865
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.62.keytab
--principal=oryx/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:45253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:38865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:08.814977 21891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:08.815306 21891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:08.815358 21891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:08.816840 21891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:02:08.816947 21891 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 07:02:08.816972 21891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:08.817004 21891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:08.817034 21891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:02:08.817065 21891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:02:08.819463 21891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:45253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:38865
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.62.keytab
--principal=oryx/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:38865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:08.819947 21891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:08.820240 21891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:08.827358 21896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:08.827359 21899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:08.827395 21897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:08.828230 21891 server_base.cc:1034] running on GCE node
I20260808 07:02:08.828828 21891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:08.829463 21891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:08.830787 21891 hybrid_clock.cc:648] HybridClock initialized: now 1786172528830783 us; error 108 us; skew 500 ppm
Aug 08 07:02:08 dist-test-slave-8tdl krb5kdc[21878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172528, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:08.834254 21891 init.cc:377] Logged in from keytab as oryx/127.13.207.62@KRBTEST.COM (short username oryx)
I20260808 07:02:08.835052 21891 webserver.cc:460] Webserver started at http://127.13.207.62:42349/ using document root <none> and password file <none>
I20260808 07:02:08.835364 21891 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:08.835714 21891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:08.836064 21891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:08.837630 21891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "befc8963088a49588ab2028974afc664"
format_stamp: "Formatted at 2026-08-08 07:02:08 on dist-test-slave-8tdl"
server_key: "0bac30fae6d7c65df8b1b08cd62beee3"
server_key_iv: "fdd8d7f49847588883917dc6260a3097"
server_key_version: "encryptionkey@0"
I20260808 07:02:08.838162 21891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "befc8963088a49588ab2028974afc664"
format_stamp: "Formatted at 2026-08-08 07:02:08 on dist-test-slave-8tdl"
server_key: "0bac30fae6d7c65df8b1b08cd62beee3"
server_key_iv: "fdd8d7f49847588883917dc6260a3097"
server_key_version: "encryptionkey@0"
I20260808 07:02:08.840124 21891 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.005s sys 0.000s
I20260808 07:02:08.841210 21907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:08.841591 21891 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:08.841727 21891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "befc8963088a49588ab2028974afc664"
format_stamp: "Formatted at 2026-08-08 07:02:08 on dist-test-slave-8tdl"
server_key: "0bac30fae6d7c65df8b1b08cd62beee3"
server_key_iv: "fdd8d7f49847588883917dc6260a3097"
server_key_version: "encryptionkey@0"
I20260808 07:02:08.842073 21891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:08.859434 21891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:08.862535 21891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:08.862782 21891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:08.870852 21891 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:38865
I20260808 07:02:08.870898 21974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:38865 every 8 connection(s)
I20260808 07:02:08.871343 21891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:02:08.872156 21975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:08.873232 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 21891
I20260808 07:02:08.873479 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:02:08.873909 14140 external_mini_cluster.cc:1428] Setting key 21861ad0ccfdec77d29b9aa6fc01c4c9
I20260808 07:02:08.876081 21975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664: Bootstrap starting.
I20260808 07:02:08.877341 21975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:08.877709 21975 log.cc:824] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:08.878631 21975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664: No bootstrap required, opened a new log
I20260808 07:02:08.881656 21975 raft_consensus.cc:348] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "befc8963088a49588ab2028974afc664" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38865 } }
I20260808 07:02:08.881915 21975 raft_consensus.cc:374] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:08.881964 21975 raft_consensus.cc:729] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: befc8963088a49588ab2028974afc664, State: Initialized, Role: FOLLOWER
I20260808 07:02:08.882133 21975 consensus_queue.cc:260] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "befc8963088a49588ab2028974afc664" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38865 } }
I20260808 07:02:08.882222 21975 raft_consensus.cc:388] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:02:08.882261 21975 raft_consensus.cc:482] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:02:08.882328 21975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 07:02:08 dist-test-slave-8tdl krb5kdc[21878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM
I20260808 07:02:08.883395 21975 raft_consensus.cc:504] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "befc8963088a49588ab2028974afc664" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38865 } }
I20260808 07:02:08.883554 21975 leader_election.cc:302] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: befc8963088a49588ab2028974afc664; no voters:
I20260808 07:02:08.883791 21975 leader_election.cc:288] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:02:08.883872 21981 raft_consensus.cc:2781] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:02:08.884138 21981 raft_consensus.cc:686] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664 [term 1 LEADER]: Becoming Leader. State: Replica: befc8963088a49588ab2028974afc664, State: Running, Role: LEADER
I20260808 07:02:08.884274 21981 consensus_queue.cc:237] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "befc8963088a49588ab2028974afc664" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38865 } }
I20260808 07:02:08.884315 21975 sys_catalog.cc:564] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:02:08.884791 21981 sys_catalog.cc:455] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664 [sys.catalog]: SysCatalogTable state changed. Reason: New leader befc8963088a49588ab2028974afc664. Latest consensus state: current_term: 1 leader_uuid: "befc8963088a49588ab2028974afc664" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "befc8963088a49588ab2028974afc664" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38865 } } }
I20260808 07:02:08.884904 21981 sys_catalog.cc:458] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:08.884804 21982 sys_catalog.cc:455] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "befc8963088a49588ab2028974afc664" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "befc8963088a49588ab2028974afc664" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38865 } } }
I20260808 07:02:08.885126 21982 sys_catalog.cc:458] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:08.885226 21986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:02:08.885841 21986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:02:08.888044 21986 catalog_manager.cc:1347] Generated new cluster ID: 6b29b8be04624c9a9730bd41d38ee09d
I20260808 07:02:08.888110 21986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:02:08.888875 21978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:08.874592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43060 (local address 127.13.207.62:38865)
0808 07:02:08.874837 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:08.874846 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:08.874882 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:02:08.875821 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:08.875830 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:08.875890 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:08.876038 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:08.877418 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:08.878529 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:08.879862 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:08.880467 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:08.880580 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:08.884073 (+ 3493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:08.884080 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:08.884080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:08.884099 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:08.886742 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:08.887242 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:08.887247 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:08.887247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:08.887406 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:08.887715 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:08.887717 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:08.887717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:08.888076 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:08.888270 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:08.888538 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 07:02:08.888702 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":133,"threads_started":1}
I20260808 07:02:08.896169 21986 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:02:08.896819 21986 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:02:08.910022 21986 catalog_manager.cc:5868] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664: Generated new TSK 0
I20260808 07:02:08.910347 21986 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.1.keytab.
Entry for principal oryx/127.13.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.1.keytab.
I20260808 07:02:08.969065 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:38865
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.1.keytab
--principal=oryx/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:45253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:09.084245 22003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:09.084523 22003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:09.084554 22003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:09.086135 22003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:09.086201 22003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:09.086257 22003 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:02:09.086292 22003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:02:09.089044 22003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:45253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.1.keytab
--principal=oryx/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.207.62:38865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:09.089323 22003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:09.089551 22003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:09.095661 22009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:09.095715 22011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:09.095821 22003 server_base.cc:1034] running on GCE node
W20260808 07:02:09.095664 22008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:09.096168 22003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:09.096489 22003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:09.097680 22003 hybrid_clock.cc:648] HybridClock initialized: now 1786172529097650 us; error 42 us; skew 500 ppm
Aug 08 07:02:09 dist-test-slave-8tdl krb5kdc[21878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172529, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:09.100963 22003 init.cc:377] Logged in from keytab as oryx/127.13.207.1@KRBTEST.COM (short username oryx)
I20260808 07:02:09.101445 22003 webserver.cc:460] Webserver started at http://127.13.207.1:43479/ using document root <none> and password file <none>
I20260808 07:02:09.101991 22003 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:09.102053 22003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:09.102164 22003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:09.103614 22003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "ec60aadcbf224a4aabffcd82f789dc66"
format_stamp: "Formatted at 2026-08-08 07:02:09 on dist-test-slave-8tdl"
server_key: "5360ee98e59a65075bc5ccca0d396901"
server_key_iv: "33a7cfba7cb5124b5ca891cfeba35046"
server_key_version: "encryptionkey@0"
I20260808 07:02:09.104058 22003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "ec60aadcbf224a4aabffcd82f789dc66"
format_stamp: "Formatted at 2026-08-08 07:02:09 on dist-test-slave-8tdl"
server_key: "5360ee98e59a65075bc5ccca0d396901"
server_key_iv: "33a7cfba7cb5124b5ca891cfeba35046"
server_key_version: "encryptionkey@0"
I20260808 07:02:09.106146 22003 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 07:02:09.107452 22018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:09.107941 22003 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:02:09.108126 22003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "ec60aadcbf224a4aabffcd82f789dc66"
format_stamp: "Formatted at 2026-08-08 07:02:09 on dist-test-slave-8tdl"
server_key: "5360ee98e59a65075bc5ccca0d396901"
server_key_iv: "33a7cfba7cb5124b5ca891cfeba35046"
server_key_version: "encryptionkey@0"
I20260808 07:02:09.108261 22003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:09.137755 22003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:09.140763 22003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:09.140941 22003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:09.141587 22003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:09.141631 22003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:09.141664 22003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:09.141703 22003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:02:09 dist-test-slave-8tdl krb5kdc[21878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172529, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM
I20260808 07:02:09.151242 22003 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:38045
I20260808 07:02:09.151665 22138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:38045 every 8 connection(s)
I20260808 07:02:09.151901 22003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:02:09.155082 21978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.142231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:53721 (local address 127.13.207.62:38865)
0808 07:02:09.142313 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:09.142317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:09.142647 (+ 330us) server_negotiation.cc:407] Connection header received
0808 07:02:09.143530 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:09.143532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:09.143546 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:09.143645 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:09.144371 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.145032 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.146966 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.147376 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.147459 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.150876 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:09.150878 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:09.150879 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:09.150891 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:09.152873 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:09.153547 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:09.153549 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:09.153549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:09.153620 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:09.154006 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:09.154007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:09.154007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:09.154159 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:09.154235 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:09.154800 (+ 565us) server_negotiation.cc:299] Negotiation successful
0808 07:02:09.154951 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 07:02:09.155361 22028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.142194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38865 (local address 127.13.207.1:53721)
0808 07:02:09.142408 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:09.142466 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:09.143313 (+ 847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:09.143759 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:09.143763 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:09.144048 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:09.144259 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.144264 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.145150 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.145151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.146841 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.146846 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.147565 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.147566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.147748 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.148401 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:09.148414 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:09.150465 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:09.153115 (+ 2650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:09.153122 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:09.153124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:09.153402 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:09.153711 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:09.153712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:09.153713 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:09.153836 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:09.154255 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:09.154262 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:09.154495 (+ 233us) client_negotiation.cc:769] Sending connection context
0808 07:02:09.154772 (+ 277us) client_negotiation.cc:240] Negotiation successful
0808 07:02:09.154978 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":111,"threads_started":1}
I20260808 07:02:09.156226 22139 heartbeater.cc:347] Connected to a master server at 127.13.207.62:38865
I20260808 07:02:09.156438 22139 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:09.156663 22139 heartbeater.cc:511] Master 127.13.207.62:38865 requested a full tablet report, sending...
I20260808 07:02:09.157567 21926 ts_manager.cc:194] Registered new tserver with Master: ec60aadcbf224a4aabffcd82f789dc66 (127.13.207.1:38045)
I20260808 07:02:09.157739 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 22003
I20260808 07:02:09.157855 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:02:09.158072 14140 external_mini_cluster.cc:1428] Setting key 794ac4b2cfb04f2d71efe6e02713432b
I20260808 07:02:09.158754 21926 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:53721
I20260808 07:02:09.165400 22145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.157425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38865 (local address 127.13.207.1:34239)
0808 07:02:09.157605 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:09.157617 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:09.157704 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:09.157867 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:09.157868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:09.158136 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:09.158231 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.158234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.159299 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.159300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.159979 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.159982 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.160468 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.160469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.160556 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.161263 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:09.161269 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:09.161657 (+ 388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:09.164132 (+ 2475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:09.164133 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:09.164133 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:09.164357 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:09.164647 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:09.164648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:09.164649 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:09.164732 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:09.165067 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:09.165068 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:09.165131 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 07:02:09.165198 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 07:02:09.165304 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":13,"thread_start_us":118,"threads_started":1}
I20260808 07:02:09.165503 21978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.157478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:34239 (local address 127.13.207.62:38865)
0808 07:02:09.157536 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:09.157538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:09.157676 (+ 138us) server_negotiation.cc:407] Connection header received
0808 07:02:09.157794 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:09.157795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:09.157807 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:09.157864 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:09.158351 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.159189 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.160117 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.160360 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.160465 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.161790 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:09.161791 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:09.161792 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:09.161803 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:09.163980 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:09.164454 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:09.164456 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:09.164457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:09.164535 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:09.164808 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:09.164809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:09.164809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:09.164981 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:09.165072 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:09.165252 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 07:02:09.165373 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
WARNING: no policy specified for oryx/127.13.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.2.keytab.
Entry for principal oryx/127.13.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.2.keytab.
I20260808 07:02:09.195848 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:38865
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.2.keytab
--principal=oryx/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:45253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:09.290969 22148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:09.291254 22148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:09.291287 22148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:09.293256 22148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:09.293344 22148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:09.293371 22148 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:02:09.293393 22148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:02:09.295434 22148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:45253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.2.keytab
--principal=oryx/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.207.62:38865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:09.295763 22148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:09.296056 22148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:09.301846 22157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:09.301868 22154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:09.301950 22153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:09.302222 22148 server_base.cc:1034] running on GCE node
I20260808 07:02:09.302461 22148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:09.303025 22148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:09.304268 22148 hybrid_clock.cc:648] HybridClock initialized: now 1786172529304250 us; error 30 us; skew 500 ppm
Aug 08 07:02:09 dist-test-slave-8tdl krb5kdc[21878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172529, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:09.307346 22148 init.cc:377] Logged in from keytab as oryx/127.13.207.2@KRBTEST.COM (short username oryx)
I20260808 07:02:09.308012 22148 webserver.cc:460] Webserver started at http://127.13.207.2:39221/ using document root <none> and password file <none>
I20260808 07:02:09.308285 22148 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:09.308367 22148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:09.308549 22148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:09.309816 22148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "ef8811953fbe41e5821e087f2e58eab0"
format_stamp: "Formatted at 2026-08-08 07:02:09 on dist-test-slave-8tdl"
server_key: "51fe8586c062c9c5691305d6def0f791"
server_key_iv: "23377a9ad6660d8d3930b00755139405"
server_key_version: "encryptionkey@0"
I20260808 07:02:09.310251 22148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "ef8811953fbe41e5821e087f2e58eab0"
format_stamp: "Formatted at 2026-08-08 07:02:09 on dist-test-slave-8tdl"
server_key: "51fe8586c062c9c5691305d6def0f791"
server_key_iv: "23377a9ad6660d8d3930b00755139405"
server_key_version: "encryptionkey@0"
I20260808 07:02:09.312160 22148 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 07:02:09.313297 22164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:09.313536 22148 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:09.313665 22148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "ef8811953fbe41e5821e087f2e58eab0"
format_stamp: "Formatted at 2026-08-08 07:02:09 on dist-test-slave-8tdl"
server_key: "51fe8586c062c9c5691305d6def0f791"
server_key_iv: "23377a9ad6660d8d3930b00755139405"
server_key_version: "encryptionkey@0"
I20260808 07:02:09.313756 22148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:09.323287 22148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:09.325963 22148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:09.326177 22148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:09.326923 22148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:09.326993 22148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:09.327030 22148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:09.327054 22148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:02:09 dist-test-slave-8tdl krb5kdc[21878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172529, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM
I20260808 07:02:09.336232 22148 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:46755
I20260808 07:02:09.336402 22287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:46755 every 8 connection(s)
I20260808 07:02:09.336752 22148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:02:09.338829 21978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.327502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:40999 (local address 127.13.207.62:38865)
0808 07:02:09.327637 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:09.327641 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:09.327824 (+ 183us) server_negotiation.cc:407] Connection header received
0808 07:02:09.328357 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:09.328358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:09.328375 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:09.328472 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:09.329360 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.330300 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.331745 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.331990 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.332096 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.335221 (+ 3125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:09.335223 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:09.335224 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:09.335233 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:09.337214 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:09.337761 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:09.337762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:09.337762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:09.337832 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:09.338102 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:09.338103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:09.338103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:09.338233 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:09.338295 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:09.338584 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 07:02:09.338667 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260808 07:02:09.339097 22173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.327465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38865 (local address 127.13.207.2:40999)
0808 07:02:09.327672 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:09.327719 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:09.328253 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:09.328622 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:09.328628 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:09.329028 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:09.329257 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.329263 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.330430 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.330432 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.331604 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.331607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.332112 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.332113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.332273 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.332983 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:09.332998 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:09.335087 (+ 2089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:09.337353 (+ 2266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:09.337359 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:09.337361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:09.337672 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:09.337915 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:09.337916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:09.337916 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:09.338035 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:09.338283 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:09.338286 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:09.338415 (+ 129us) client_negotiation.cc:769] Sending connection context
0808 07:02:09.338560 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 07:02:09.338799 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":100,"threads_started":1}
I20260808 07:02:09.339762 22288 heartbeater.cc:347] Connected to a master server at 127.13.207.62:38865
I20260808 07:02:09.339859 22288 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:09.340158 22288 heartbeater.cc:511] Master 127.13.207.62:38865 requested a full tablet report, sending...
I20260808 07:02:09.340626 21924 ts_manager.cc:194] Registered new tserver with Master: ef8811953fbe41e5821e087f2e58eab0 (127.13.207.2:46755)
I20260808 07:02:09.341259 21924 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:40999
I20260808 07:02:09.343588 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 22148
I20260808 07:02:09.343746 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:02:09.344257 14140 external_mini_cluster.cc:1428] Setting key 7bd4afacea48e3ef43392ffcf4daddbb
I20260808 07:02:09.348939 21978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.341171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:51811 (local address 127.13.207.62:38865)
0808 07:02:09.341219 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:09.341222 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:09.341354 (+ 132us) server_negotiation.cc:407] Connection header received
0808 07:02:09.341527 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:09.341528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:09.341537 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:09.341599 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:09.341974 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.342575 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.343476 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.343705 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.343799 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.345427 (+ 1628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:09.345429 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:09.345429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:09.345438 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:09.347371 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:09.347900 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:09.347901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:09.347901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:09.347961 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:09.348264 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:09.348265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:09.348265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:09.348456 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:09.348538 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:09.348778 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 07:02:09.348864 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:02:09.349052 22294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.341110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38865 (local address 127.13.207.2:51811)
0808 07:02:09.341264 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:09.341278 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:09.341460 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:09.341601 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:09.341602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:09.341807 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:09.341879 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.341880 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.342736 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.342738 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.343373 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.343375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.343801 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.343803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.343891 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.344846 (+ 955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:09.344852 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:09.345327 (+ 475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:09.347496 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:09.347498 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:09.347498 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:09.347788 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:09.348013 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:09.348014 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:09.348015 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:09.348102 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:09.348555 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:09.348556 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:09.348636 (+ 80us) client_negotiation.cc:769] Sending connection context
0808 07:02:09.348720 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 07:02:09.348862 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
WARNING: no policy specified for oryx/127.13.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.3.keytab.
Entry for principal oryx/127.13.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.3.keytab.
I20260808 07:02:09.383598 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:38865
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.3.keytab
--principal=oryx/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:45253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:09.486075 22297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:09.486351 22297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:09.486394 22297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:09.487761 22297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:09.487843 22297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:09.487890 22297 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:02:09.487954 22297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:02:09.489657 22297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:45253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.3.keytab
--principal=oryx/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.207.62:38865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:09.489977 22297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:09.490242 22297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:09.496397 22303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:09.496408 22306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:09.496827 22304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:09.496908 22297 server_base.cc:1034] running on GCE node
I20260808 07:02:09.497273 22297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:09.497733 22297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:09.499056 22297 hybrid_clock.cc:648] HybridClock initialized: now 1786172529498984 us; error 87 us; skew 500 ppm
Aug 08 07:02:09 dist-test-slave-8tdl krb5kdc[21878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172529, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:09.502329 22297 init.cc:377] Logged in from keytab as oryx/127.13.207.3@KRBTEST.COM (short username oryx)
I20260808 07:02:09.503043 22297 webserver.cc:460] Webserver started at http://127.13.207.3:33149/ using document root <none> and password file <none>
I20260808 07:02:09.503321 22297 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:09.503412 22297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:09.503585 22297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:09.504864 22297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "41e555b6549a45168d2b12376f64daea"
format_stamp: "Formatted at 2026-08-08 07:02:09 on dist-test-slave-8tdl"
server_key: "731464ee016d114d2b1f9550a992257d"
server_key_iv: "624fc04369f1770b5a52e1f3de4b46ba"
server_key_version: "encryptionkey@0"
I20260808 07:02:09.505286 22297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "41e555b6549a45168d2b12376f64daea"
format_stamp: "Formatted at 2026-08-08 07:02:09 on dist-test-slave-8tdl"
server_key: "731464ee016d114d2b1f9550a992257d"
server_key_iv: "624fc04369f1770b5a52e1f3de4b46ba"
server_key_version: "encryptionkey@0"
I20260808 07:02:09.507048 22297 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 07:02:09.508158 22313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:09.508374 22297 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:09.508522 22297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "41e555b6549a45168d2b12376f64daea"
format_stamp: "Formatted at 2026-08-08 07:02:09 on dist-test-slave-8tdl"
server_key: "731464ee016d114d2b1f9550a992257d"
server_key_iv: "624fc04369f1770b5a52e1f3de4b46ba"
server_key_version: "encryptionkey@0"
I20260808 07:02:09.508659 22297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:09.521746 22297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:09.526063 22297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:09.526284 22297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:09.527213 22297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:09.527477 22297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:09.527681 22297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:09.527743 22297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:09.536612 22297 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:40383
I20260808 07:02:09.536705 22433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:40383 every 8 connection(s)
I20260808 07:02:09.537106 22297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:02:09 dist-test-slave-8tdl krb5kdc[21878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172529, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM
I20260808 07:02:09.541316 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 22297
I20260808 07:02:09.541491 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:02:09.541729 14140 external_mini_cluster.cc:1428] Setting key 593e4ec42b473b670135bf7a83b80f57
I20260808 07:02:09.542752 21978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.527816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:41955 (local address 127.13.207.62:38865)
0808 07:02:09.527883 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:09.527885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:09.528213 (+ 328us) server_negotiation.cc:407] Connection header received
0808 07:02:09.529534 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:09.529535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:09.529553 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:09.529742 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:09.531619 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.532490 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.534379 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.534806 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.534913 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.538853 (+ 3940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:09.538855 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:09.538855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:09.538872 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:09.540500 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:09.541021 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:09.541022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:09.541022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:09.541134 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:09.541649 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:09.541651 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:09.541651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:09.541833 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:09.541926 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:09.542507 (+ 581us) server_negotiation.cc:299] Negotiation successful
0808 07:02:09.542618 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 07:02:09.542990 22321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.527841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38865 (local address 127.13.207.3:41955)
0808 07:02:09.528055 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:09.528101 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:09.529261 (+ 1160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:09.530188 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:09.530268 (+ 80us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:09.530911 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:09.531323 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.531330 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.532615 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.532617 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.534250 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.534255 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.534933 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.534934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.535328 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.536681 (+ 1353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:09.536693 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:09.538667 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:09.540624 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:09.540632 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:09.540634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:09.540905 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:09.541317 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:09.541318 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:09.541318 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:09.541543 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:09.542052 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:09.542060 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:09.542250 (+ 190us) client_negotiation.cc:769] Sending connection context
0808 07:02:09.542435 (+ 185us) client_negotiation.cc:240] Negotiation successful
0808 07:02:09.542645 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":94,"threads_started":1}
I20260808 07:02:09.543507 22434 heartbeater.cc:347] Connected to a master server at 127.13.207.62:38865
I20260808 07:02:09.543608 22434 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:09.543843 22434 heartbeater.cc:511] Master 127.13.207.62:38865 requested a full tablet report, sending...
I20260808 07:02:09.544628 21924 ts_manager.cc:194] Registered new tserver with Master: 41e555b6549a45168d2b12376f64daea (127.13.207.3:40383)
I20260808 07:02:09.545105 21924 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:41955
I20260808 07:02:09.552064 21978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.544836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:41061 (local address 127.13.207.62:38865)
0808 07:02:09.544926 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:09.544931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:09.545510 (+ 579us) server_negotiation.cc:407] Connection header received
0808 07:02:09.545541 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:09.545542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:09.545554 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:09.545637 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:09.545994 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.546533 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.547319 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.547536 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.547636 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.548683 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:09.548684 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:09.548685 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:09.548692 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:09.550321 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:09.550997 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:09.550999 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:09.551000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:09.551075 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:09.551317 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:09.551318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:09.551319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:09.551460 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:09.551553 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:09.551752 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 07:02:09.551840 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 07:02:09.552085 22440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.544763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38865 (local address 127.13.207.3:41061)
0808 07:02:09.545316 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:09.545329 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:09.545414 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:09.545640 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:09.545640 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:09.545829 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:09.545895 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.545897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.546632 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.546633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.547227 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.547229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.547640 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.547640 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.547719 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.548215 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:09.548220 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:09.548553 (+ 333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:09.550580 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:09.550581 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:09.550582 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:09.550899 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:09.551168 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:09.551169 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:09.551170 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:09.551245 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:09.551549 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:09.551550 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:09.551609 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 07:02:09.551701 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 07:02:09.551840 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":87,"threads_started":1}
I20260808 07:02:09.554658 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 07:02:09 dist-test-slave-8tdl krb5kdc[21878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172529, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:02:09 dist-test-slave-8tdl krb5kdc[21878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:09 dist-test-slave-8tdl krb5kdc[21878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
W20260808 07:02:09.585970 21978 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:09.577847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43070 (local address 127.13.207.62:38865)
0808 07:02:09.578062 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:09.578064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:09.578092 (+ 28us) server_negotiation.cc:407] Connection header received
0808 07:02:09.578128 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:09.578129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:09.578140 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:09.578231 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:09.579087 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.579895 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.580877 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.581088 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.581168 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.585860 (+ 4692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43070: BlockingRecv error: recv got EOF from 127.0.0.1:43070 (error 108)
Metrics: {"server-negotiator.queue_time_us":178}
Aug 08 07:02:09 dist-test-slave-8tdl krb5kdc[21878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172529, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM
I20260808 07:02:09.597724 21978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.587383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43080 (local address 127.13.207.62:38865)
0808 07:02:09.587525 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:09.587528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:09.587542 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:02:09.587576 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:09.587577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:09.587591 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:09.587674 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:09.588658 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.589359 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.590481 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.590833 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.590939 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.594150 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:09.594153 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:09.594153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:09.594163 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:09.595788 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:09.596295 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:09.596296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:09.596296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:09.596371 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:09.596928 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:09.596929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:09.596929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:09.597065 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:09.597161 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:09.597557 (+ 396us) server_negotiation.cc:299] Negotiation successful
0808 07:02:09.597642 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260808 07:02:09.600683 21924 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43080:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:02:09.602445 21924 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:02:09.611665 22459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.606703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33892 (local address 127.13.207.2:46755)
0808 07:02:09.607206 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:09.607210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:09.607220 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:02:09.607256 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:09.607258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:09.607272 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:09.607350 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:09.607563 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.608680 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.610393 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.611132 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.611217 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.611302 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:09.611494 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 07:02:09.611538 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":92,"threads_started":1}
I20260808 07:02:09.611699 22458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.606566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:46755 (local address 127.0.0.1:33892)
0808 07:02:09.606980 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:09.606992 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:09.607057 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:09.607419 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:09.607419 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:09.607421 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:09.607466 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.607468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.608780 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.608781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.610280 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.610283 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.611238 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.611239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.611345 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.611364 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 07:02:09.611456 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 07:02:09.611533 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":126,"thread_start_us":158,"threads_started":1}
I20260808 07:02:09.612394 22461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.606860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:38045 (local address 127.0.0.1:52822)
0808 07:02:09.607404 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:09.607416 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:09.607471 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:09.607710 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:09.607711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:09.607713 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:09.607940 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.607943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.609286 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.609287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.610860 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.610862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.612075 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.612076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.612156 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.612161 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:02:09.612253 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 07:02:09.612306 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"mutex_wait_us":5,"thread_start_us":87,"threads_started":1}
I20260808 07:02:09.612425 22462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.607153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52822 (local address 127.13.207.1:38045)
0808 07:02:09.607320 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:09.607325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:09.607536 (+ 211us) server_negotiation.cc:407] Connection header received
0808 07:02:09.607588 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:09.607591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:09.607609 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:09.607709 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:09.608147 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.609122 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.610954 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.611794 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.611872 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.612032 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:09.612271 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 07:02:09.612317 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":60,"threads_started":1}
I20260808 07:02:09.612689 22457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.606492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:40383 (local address 127.0.0.1:60512)
0808 07:02:09.606734 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:09.606785 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:09.606951 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:09.607332 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:09.607336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:09.607338 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:09.607422 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.607424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.608814 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.608814 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.610810 (+ 1996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.610814 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.612438 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.612439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.612507 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.612510 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:02:09.612562 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:02:09.612609 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":363,"thread_start_us":88,"threads_started":1}
I20260808 07:02:09.612936 22460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.606739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60512 (local address 127.13.207.3:40383)
0808 07:02:09.606956 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:09.606962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:09.606977 (+ 15us) server_negotiation.cc:407] Connection header received
0808 07:02:09.607129 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:09.607133 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:09.607152 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:09.607237 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:09.607514 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.608680 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.610940 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.611928 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.612010 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.612085 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:09.612780 (+ 695us) server_negotiation.cc:299] Negotiation successful
0808 07:02:09.612838 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":103,"threads_started":1}
I20260808 07:02:09.612803 22217 tablet_service.cc:1467] Processing CreateTablet for tablet 14461b57e3944af5acf8cdd0fc3fcf4d (DEFAULT_TABLE table=test-table [id=a1fb982cc2fc4a5185274b121f6080b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:09.613263 22217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14461b57e3944af5acf8cdd0fc3fcf4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:09.613654 22069 tablet_service.cc:1467] Processing CreateTablet for tablet 14461b57e3944af5acf8cdd0fc3fcf4d (DEFAULT_TABLE table=test-table [id=a1fb982cc2fc4a5185274b121f6080b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:09.614069 22069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14461b57e3944af5acf8cdd0fc3fcf4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:09.614486 22364 tablet_service.cc:1467] Processing CreateTablet for tablet 14461b57e3944af5acf8cdd0fc3fcf4d (DEFAULT_TABLE table=test-table [id=a1fb982cc2fc4a5185274b121f6080b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:09.615001 22364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14461b57e3944af5acf8cdd0fc3fcf4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:09.618196 22463 tablet_bootstrap.cc:492] T 14461b57e3944af5acf8cdd0fc3fcf4d P ef8811953fbe41e5821e087f2e58eab0: Bootstrap starting.
I20260808 07:02:09.618775 22464 tablet_bootstrap.cc:492] T 14461b57e3944af5acf8cdd0fc3fcf4d P ec60aadcbf224a4aabffcd82f789dc66: Bootstrap starting.
I20260808 07:02:09.619340 22463 tablet_bootstrap.cc:654] T 14461b57e3944af5acf8cdd0fc3fcf4d P ef8811953fbe41e5821e087f2e58eab0: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:09.619580 22465 tablet_bootstrap.cc:492] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea: Bootstrap starting.
I20260808 07:02:09.619820 22464 tablet_bootstrap.cc:654] T 14461b57e3944af5acf8cdd0fc3fcf4d P ec60aadcbf224a4aabffcd82f789dc66: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:09.620082 22463 log.cc:824] T 14461b57e3944af5acf8cdd0fc3fcf4d P ef8811953fbe41e5821e087f2e58eab0: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:09.620169 22464 log.cc:824] T 14461b57e3944af5acf8cdd0fc3fcf4d P ec60aadcbf224a4aabffcd82f789dc66: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:09.620297 22465 tablet_bootstrap.cc:654] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:09.620671 22465 log.cc:824] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:09.621127 22464 tablet_bootstrap.cc:492] T 14461b57e3944af5acf8cdd0fc3fcf4d P ec60aadcbf224a4aabffcd82f789dc66: No bootstrap required, opened a new log
I20260808 07:02:09.621232 22464 ts_tablet_manager.cc:1397] T 14461b57e3944af5acf8cdd0fc3fcf4d P ec60aadcbf224a4aabffcd82f789dc66: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:02:09.621302 22463 tablet_bootstrap.cc:492] T 14461b57e3944af5acf8cdd0fc3fcf4d P ef8811953fbe41e5821e087f2e58eab0: No bootstrap required, opened a new log
I20260808 07:02:09.621381 22463 ts_tablet_manager.cc:1397] T 14461b57e3944af5acf8cdd0fc3fcf4d P ef8811953fbe41e5821e087f2e58eab0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:02:09.621562 22465 tablet_bootstrap.cc:492] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea: No bootstrap required, opened a new log
I20260808 07:02:09.621641 22465 ts_tablet_manager.cc:1397] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:02:09.623075 22463 raft_consensus.cc:348] T 14461b57e3944af5acf8cdd0fc3fcf4d P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } } peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } }
I20260808 07:02:09.623278 22463 raft_consensus.cc:374] T 14461b57e3944af5acf8cdd0fc3fcf4d P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:09.623318 22463 raft_consensus.cc:729] T 14461b57e3944af5acf8cdd0fc3fcf4d P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef8811953fbe41e5821e087f2e58eab0, State: Initialized, Role: FOLLOWER
I20260808 07:02:09.623479 22463 consensus_queue.cc:260] T 14461b57e3944af5acf8cdd0fc3fcf4d P ef8811953fbe41e5821e087f2e58eab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } } peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } }
I20260808 07:02:09.623831 22463 ts_tablet_manager.cc:1428] T 14461b57e3944af5acf8cdd0fc3fcf4d P ef8811953fbe41e5821e087f2e58eab0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:02:09.623791 22464 raft_consensus.cc:348] T 14461b57e3944af5acf8cdd0fc3fcf4d P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } } peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } }
I20260808 07:02:09.623960 22464 raft_consensus.cc:374] T 14461b57e3944af5acf8cdd0fc3fcf4d P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:09.623998 22464 raft_consensus.cc:729] T 14461b57e3944af5acf8cdd0fc3fcf4d P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec60aadcbf224a4aabffcd82f789dc66, State: Initialized, Role: FOLLOWER
I20260808 07:02:09.624042 22288 heartbeater.cc:503] Master 127.13.207.62:38865 was elected leader, sending a full tablet report...
I20260808 07:02:09.624123 22464 consensus_queue.cc:260] T 14461b57e3944af5acf8cdd0fc3fcf4d P ec60aadcbf224a4aabffcd82f789dc66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } } peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } }
I20260808 07:02:09.624110 22465 raft_consensus.cc:348] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } } peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } }
I20260808 07:02:09.624274 22465 raft_consensus.cc:374] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:09.624315 22465 raft_consensus.cc:729] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41e555b6549a45168d2b12376f64daea, State: Initialized, Role: FOLLOWER
I20260808 07:02:09.624402 22464 ts_tablet_manager.cc:1428] T 14461b57e3944af5acf8cdd0fc3fcf4d P ec60aadcbf224a4aabffcd82f789dc66: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 07:02:09.624446 22465 consensus_queue.cc:260] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } } peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } }
I20260808 07:02:09.624524 22139 heartbeater.cc:503] Master 127.13.207.62:38865 was elected leader, sending a full tablet report...
I20260808 07:02:09.624728 22465 ts_tablet_manager.cc:1428] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:02:09.624897 22434 heartbeater.cc:503] Master 127.13.207.62:38865 was elected leader, sending a full tablet report...
W20260808 07:02:09.652944 22140 tablet.cc:2369] T 14461b57e3944af5acf8cdd0fc3fcf4d P ec60aadcbf224a4aabffcd82f789dc66: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:02:09.733250 22472 raft_consensus.cc:482] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:02:09.733397 22472 raft_consensus.cc:504] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } } peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } }
I20260808 07:02:09.733809 22472 leader_election.cc:288] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef8811953fbe41e5821e087f2e58eab0 (127.13.207.2:46755), ec60aadcbf224a4aabffcd82f789dc66 (127.13.207.1:38045)
I20260808 07:02:09.738976 22462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.734463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:59419 (local address 127.13.207.1:38045)
0808 07:02:09.734541 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:09.734543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:09.734837 (+ 294us) server_negotiation.cc:407] Connection header received
0808 07:02:09.734870 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:09.734871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:09.734882 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:09.734942 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:09.735132 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.735806 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.737477 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.738324 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.738466 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.738480 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:09.738843 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 07:02:09.738892 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 07:02:09.738986 22459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.734317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:36943 (local address 127.13.207.2:46755)
0808 07:02:09.734386 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:09.734388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:09.734402 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:02:09.734519 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:09.734520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:09.734533 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:09.734602 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:09.734834 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.735596 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.737490 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.738309 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.738498 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.738509 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:09.738843 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 07:02:09.738898 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:02:09.739008 22321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.734171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:46755 (local address 127.13.207.3:36943)
0808 07:02:09.734314 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:09.734330 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:09.734402 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:09.734609 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:09.734610 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:09.734611 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:09.734737 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.734740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.735967 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.735970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.737311 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.737316 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.738555 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.738556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.738729 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.738733 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:02:09.738814 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 07:02:09.738874 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":17}
I20260808 07:02:09.739017 22473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.734310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:38045 (local address 127.13.207.3:59419)
0808 07:02:09.734522 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:09.734530 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:09.734591 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:09.734949 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:09.734949 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:09.734952 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:09.735036 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.735038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.735978 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.735978 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.737311 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.737316 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.738553 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.738559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.738662 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.738667 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:02:09.738833 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 07:02:09.738899 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":32,"thread_start_us":110,"threads_started":1}
I20260808 07:02:09.739626 22238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14461b57e3944af5acf8cdd0fc3fcf4d" candidate_uuid: "41e555b6549a45168d2b12376f64daea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef8811953fbe41e5821e087f2e58eab0" is_pre_election: true
I20260808 07:02:09.739612 22089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14461b57e3944af5acf8cdd0fc3fcf4d" candidate_uuid: "41e555b6549a45168d2b12376f64daea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec60aadcbf224a4aabffcd82f789dc66" is_pre_election: true
I20260808 07:02:09.739830 22238 raft_consensus.cc:2454] T 14461b57e3944af5acf8cdd0fc3fcf4d P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41e555b6549a45168d2b12376f64daea in term 0.
I20260808 07:02:09.739835 22089 raft_consensus.cc:2454] T 14461b57e3944af5acf8cdd0fc3fcf4d P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41e555b6549a45168d2b12376f64daea in term 0.
I20260808 07:02:09.740125 22317 leader_election.cc:302] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41e555b6549a45168d2b12376f64daea, ef8811953fbe41e5821e087f2e58eab0; no voters:
I20260808 07:02:09.740290 22472 raft_consensus.cc:2781] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:02:09.740404 22472 raft_consensus.cc:482] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:02:09.740439 22472 raft_consensus.cc:3037] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:09.741619 22472 raft_consensus.cc:504] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } } peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } }
I20260808 07:02:09.741811 22472 leader_election.cc:288] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea [CANDIDATE]: Term 1 election: Requested vote from peers ef8811953fbe41e5821e087f2e58eab0 (127.13.207.2:46755), ec60aadcbf224a4aabffcd82f789dc66 (127.13.207.1:38045)
I20260808 07:02:09.742133 22238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14461b57e3944af5acf8cdd0fc3fcf4d" candidate_uuid: "41e555b6549a45168d2b12376f64daea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef8811953fbe41e5821e087f2e58eab0"
I20260808 07:02:09.742130 22089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14461b57e3944af5acf8cdd0fc3fcf4d" candidate_uuid: "41e555b6549a45168d2b12376f64daea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec60aadcbf224a4aabffcd82f789dc66"
I20260808 07:02:09.742242 22089 raft_consensus.cc:3037] T 14461b57e3944af5acf8cdd0fc3fcf4d P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:09.742251 22238 raft_consensus.cc:3037] T 14461b57e3944af5acf8cdd0fc3fcf4d P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:09.743278 22238 raft_consensus.cc:2454] T 14461b57e3944af5acf8cdd0fc3fcf4d P ef8811953fbe41e5821e087f2e58eab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41e555b6549a45168d2b12376f64daea in term 1.
I20260808 07:02:09.743278 22089 raft_consensus.cc:2454] T 14461b57e3944af5acf8cdd0fc3fcf4d P ec60aadcbf224a4aabffcd82f789dc66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41e555b6549a45168d2b12376f64daea in term 1.
I20260808 07:02:09.743505 22317 leader_election.cc:302] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41e555b6549a45168d2b12376f64daea, ef8811953fbe41e5821e087f2e58eab0; no voters:
I20260808 07:02:09.743649 22472 raft_consensus.cc:2781] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:02:09.743848 22472 raft_consensus.cc:686] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea [term 1 LEADER]: Becoming Leader. State: Replica: 41e555b6549a45168d2b12376f64daea, State: Running, Role: LEADER
I20260808 07:02:09.743960 22472 consensus_queue.cc:237] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } } peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } }
I20260808 07:02:09.745075 21924 catalog_manager.cc:5495] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea reported cstate change: term changed from 0 to 1, leader changed from <none> to 41e555b6549a45168d2b12376f64daea (127.13.207.3). New cstate: current_term: 1 leader_uuid: "41e555b6549a45168d2b12376f64daea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } health_report { overall_health: UNKNOWN } } }
I20260808 07:02:09.766162 22460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.762634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60522 (local address 127.13.207.3:40383)
0808 07:02:09.762718 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:09.762721 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:09.762795 (+ 74us) server_negotiation.cc:407] Connection header received
0808 07:02:09.762875 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:09.762876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:09.762896 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:09.763016 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:02:09.763284 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.763902 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.764779 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.764975 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.765167 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.765616 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:02:09.765729 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:02:09.765809 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:09.766048 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 07:02:09.766093 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 07:02:09.768631 22238 raft_consensus.cc:1261] T 14461b57e3944af5acf8cdd0fc3fcf4d P ef8811953fbe41e5821e087f2e58eab0 [term 1 FOLLOWER]: Refusing update from remote peer 41e555b6549a45168d2b12376f64daea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:09.768697 22089 raft_consensus.cc:1261] T 14461b57e3944af5acf8cdd0fc3fcf4d P ec60aadcbf224a4aabffcd82f789dc66 [term 1 FOLLOWER]: Refusing update from remote peer 41e555b6549a45168d2b12376f64daea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:09.769068 22474 consensus_queue.cc:1035] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:09.769237 22472 consensus_queue.cc:1035] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:09.772893 22479 mvcc.cc:205] Tried to move back new op lower bound from 7316162681928089600 to 7316162681831862272. Current Snapshot: MvccSnapshot[applied={T|T < 7316162681928089600}]
I20260808 07:02:09.773463 22478 mvcc.cc:205] Tried to move back new op lower bound from 7316162681928089600 to 7316162681831862272. Current Snapshot: MvccSnapshot[applied={T|T < 7316162681928089600}]
I20260808 07:02:09.775964 21923 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:02:09.776100 21923 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:02:09.777081 21923 catalog_manager.cc:5782] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664: Sending DeleteTablet for 3 replicas of tablet 14461b57e3944af5acf8cdd0fc3fcf4d
I20260808 07:02:09.777498 22069 tablet_service.cc:1514] Processing DeleteTablet for tablet 14461b57e3944af5acf8cdd0fc3fcf4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:09 UTC) from {username='oryx', principal='oryx/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:52822
I20260808 07:02:09.777513 22217 tablet_service.cc:1514] Processing DeleteTablet for tablet 14461b57e3944af5acf8cdd0fc3fcf4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:09 UTC) from {username='oryx', principal='oryx/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:33892
I20260808 07:02:09.777889 22364 tablet_service.cc:1514] Processing DeleteTablet for tablet 14461b57e3944af5acf8cdd0fc3fcf4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:09 UTC) from {username='oryx', principal='oryx/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:60512
I20260808 07:02:09.778113 21923 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43080:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 07:02:09.778218 22491 tablet_replica.cc:330] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea: stopping tablet replica
W20260808 07:02:09.778292 21923 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:02:09.778327 22491 raft_consensus.cc:2229] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:02:09.778461 22491 raft_consensus.cc:2258] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:09.779642 22491 ts_tablet_manager.cc:1905] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:09.779748 22364 tablet_service.cc:1467] Processing CreateTablet for tablet 65839adf20604c2888e110a810386edd (DEFAULT_TABLE table=test-table [id=ebb1abea234448c7b66d32f77f1615c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:09.779879 22364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65839adf20604c2888e110a810386edd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:09.779940 22069 tablet_service.cc:1467] Processing CreateTablet for tablet 65839adf20604c2888e110a810386edd (DEFAULT_TABLE table=test-table [id=ebb1abea234448c7b66d32f77f1615c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:09.780081 22069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65839adf20604c2888e110a810386edd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:09.781507 22217 tablet_service.cc:1467] Processing CreateTablet for tablet 65839adf20604c2888e110a810386edd (DEFAULT_TABLE table=test-table [id=ebb1abea234448c7b66d32f77f1615c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:09.781674 22217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65839adf20604c2888e110a810386edd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:09.781754 22464 tablet_bootstrap.cc:492] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66: Bootstrap starting.
I20260808 07:02:09.782171 22464 tablet_bootstrap.cc:654] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:09.782755 22491 ts_tablet_manager.cc:1918] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:02:09.782852 22491 log.cc:1196] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/14461b57e3944af5acf8cdd0fc3fcf4d
I20260808 07:02:09.783217 22464 tablet_bootstrap.cc:492] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66: No bootstrap required, opened a new log
I20260808 07:02:09.783277 22464 ts_tablet_manager.cc:1397] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:02:09.783533 22465 tablet_bootstrap.cc:492] T 65839adf20604c2888e110a810386edd P 41e555b6549a45168d2b12376f64daea: Bootstrap starting.
I20260808 07:02:09.783547 22464 raft_consensus.cc:348] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:09.783705 22464 raft_consensus.cc:374] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:09.783766 22464 raft_consensus.cc:729] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec60aadcbf224a4aabffcd82f789dc66, State: Initialized, Role: FOLLOWER
I20260808 07:02:09.783840 22464 consensus_queue.cc:260] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:09.783996 22464 ts_tablet_manager.cc:1428] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:09.784119 22465 tablet_bootstrap.cc:654] T 65839adf20604c2888e110a810386edd P 41e555b6549a45168d2b12376f64daea: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:09.784324 22491 ts_tablet_manager.cc:1939] T 14461b57e3944af5acf8cdd0fc3fcf4d P 41e555b6549a45168d2b12376f64daea: Deleting consensus metadata
I20260808 07:02:09.784799 22463 tablet_bootstrap.cc:492] T 65839adf20604c2888e110a810386edd P ef8811953fbe41e5821e087f2e58eab0: Bootstrap starting.
I20260808 07:02:09.785147 21909 catalog_manager.cc:4841] TS 41e555b6549a45168d2b12376f64daea (127.13.207.3:40383): tablet 14461b57e3944af5acf8cdd0fc3fcf4d (table test-table [id=a1fb982cc2fc4a5185274b121f6080b6]) successfully deleted
I20260808 07:02:09.785341 22463 tablet_bootstrap.cc:654] T 65839adf20604c2888e110a810386edd P ef8811953fbe41e5821e087f2e58eab0: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:09.786422 22463 tablet_bootstrap.cc:492] T 65839adf20604c2888e110a810386edd P ef8811953fbe41e5821e087f2e58eab0: No bootstrap required, opened a new log
I20260808 07:02:09.786736 22463 ts_tablet_manager.cc:1397] T 65839adf20604c2888e110a810386edd P ef8811953fbe41e5821e087f2e58eab0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260808 07:02:09.787024 22463 raft_consensus.cc:348] T 65839adf20604c2888e110a810386edd P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:09.787111 22463 raft_consensus.cc:374] T 65839adf20604c2888e110a810386edd P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:09.787127 22463 raft_consensus.cc:729] T 65839adf20604c2888e110a810386edd P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef8811953fbe41e5821e087f2e58eab0, State: Initialized, Role: FOLLOWER
I20260808 07:02:09.787192 22463 consensus_queue.cc:260] T 65839adf20604c2888e110a810386edd P ef8811953fbe41e5821e087f2e58eab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:09.787292 22463 ts_tablet_manager.cc:1428] T 65839adf20604c2888e110a810386edd P ef8811953fbe41e5821e087f2e58eab0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:09.787809 22465 tablet_bootstrap.cc:492] T 65839adf20604c2888e110a810386edd P 41e555b6549a45168d2b12376f64daea: No bootstrap required, opened a new log
I20260808 07:02:09.787878 22465 ts_tablet_manager.cc:1397] T 65839adf20604c2888e110a810386edd P 41e555b6549a45168d2b12376f64daea: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 07:02:09.788014 22465 raft_consensus.cc:348] T 65839adf20604c2888e110a810386edd P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:09.788072 22465 raft_consensus.cc:374] T 65839adf20604c2888e110a810386edd P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:09.788087 22465 raft_consensus.cc:729] T 65839adf20604c2888e110a810386edd P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41e555b6549a45168d2b12376f64daea, State: Initialized, Role: FOLLOWER
I20260808 07:02:09.788146 22465 consensus_queue.cc:260] T 65839adf20604c2888e110a810386edd P 41e555b6549a45168d2b12376f64daea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:09.788249 22465 ts_tablet_manager.cc:1428] T 65839adf20604c2888e110a810386edd P 41e555b6549a45168d2b12376f64daea: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:09.788384 22490 tablet_replica.cc:330] T 14461b57e3944af5acf8cdd0fc3fcf4d P ef8811953fbe41e5821e087f2e58eab0: stopping tablet replica
I20260808 07:02:09.788842 22490 raft_consensus.cc:2229] T 14461b57e3944af5acf8cdd0fc3fcf4d P ef8811953fbe41e5821e087f2e58eab0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:02:09.789168 22490 raft_consensus.cc:2258] T 14461b57e3944af5acf8cdd0fc3fcf4d P ef8811953fbe41e5821e087f2e58eab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:09.789726 22489 tablet_replica.cc:330] T 14461b57e3944af5acf8cdd0fc3fcf4d P ec60aadcbf224a4aabffcd82f789dc66: stopping tablet replica
I20260808 07:02:09.789866 22489 raft_consensus.cc:2229] T 14461b57e3944af5acf8cdd0fc3fcf4d P ec60aadcbf224a4aabffcd82f789dc66 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:02:09.789973 22489 raft_consensus.cc:2258] T 14461b57e3944af5acf8cdd0fc3fcf4d P ec60aadcbf224a4aabffcd82f789dc66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:09.790311 22490 ts_tablet_manager.cc:1905] T 14461b57e3944af5acf8cdd0fc3fcf4d P ef8811953fbe41e5821e087f2e58eab0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:09.790421 22489 ts_tablet_manager.cc:1905] T 14461b57e3944af5acf8cdd0fc3fcf4d P ec60aadcbf224a4aabffcd82f789dc66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:09.792989 22489 ts_tablet_manager.cc:1918] T 14461b57e3944af5acf8cdd0fc3fcf4d P ec60aadcbf224a4aabffcd82f789dc66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:02:09.792990 22490 ts_tablet_manager.cc:1918] T 14461b57e3944af5acf8cdd0fc3fcf4d P ef8811953fbe41e5821e087f2e58eab0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:02:09.793081 22489 log.cc:1196] T 14461b57e3944af5acf8cdd0fc3fcf4d P ec60aadcbf224a4aabffcd82f789dc66: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-0/wal/wals/14461b57e3944af5acf8cdd0fc3fcf4d
I20260808 07:02:09.793227 22490 log.cc:1196] T 14461b57e3944af5acf8cdd0fc3fcf4d P ef8811953fbe41e5821e087f2e58eab0: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-1/wal/wals/14461b57e3944af5acf8cdd0fc3fcf4d
I20260808 07:02:09.793334 22489 ts_tablet_manager.cc:1939] T 14461b57e3944af5acf8cdd0fc3fcf4d P ec60aadcbf224a4aabffcd82f789dc66: Deleting consensus metadata
I20260808 07:02:09.793771 22490 ts_tablet_manager.cc:1939] T 14461b57e3944af5acf8cdd0fc3fcf4d P ef8811953fbe41e5821e087f2e58eab0: Deleting consensus metadata
I20260808 07:02:09.793754 21911 catalog_manager.cc:4841] TS ec60aadcbf224a4aabffcd82f789dc66 (127.13.207.1:38045): tablet 14461b57e3944af5acf8cdd0fc3fcf4d (table test-table [id=a1fb982cc2fc4a5185274b121f6080b6]) successfully deleted
I20260808 07:02:09.794394 21911 catalog_manager.cc:4841] TS ef8811953fbe41e5821e087f2e58eab0 (127.13.207.2:46755): tablet 14461b57e3944af5acf8cdd0fc3fcf4d (table test-table [id=a1fb982cc2fc4a5185274b121f6080b6]) successfully deleted
W20260808 07:02:09.837836 22289 tablet.cc:2369] T 65839adf20604c2888e110a810386edd P ef8811953fbe41e5821e087f2e58eab0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:02:09.949105 22470 raft_consensus.cc:482] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:02:09.949226 22470 raft_consensus.cc:504] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:09.949617 22470 leader_election.cc:288] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef8811953fbe41e5821e087f2e58eab0 (127.13.207.2:46755), 41e555b6549a45168d2b12376f64daea (127.13.207.3:40383)
I20260808 07:02:09.953146 22496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.949880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:46755 (local address 127.13.207.1:60691)
0808 07:02:09.950217 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:09.950227 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:09.950311 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:09.950472 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:09.950473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:09.950475 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:09.950569 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.950571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.951350 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.951351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.952230 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.952233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.952827 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.952828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.952915 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.952918 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:02:09.952989 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 07:02:09.953036 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":171,"thread_start_us":64,"threads_started":1}
I20260808 07:02:09.953121 22459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.949998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:60691 (local address 127.13.207.2:46755)
0808 07:02:09.950061 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:09.950063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:09.950299 (+ 236us) server_negotiation.cc:407] Connection header received
0808 07:02:09.950373 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:09.950374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:09.950384 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:09.950466 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:09.950644 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.951245 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.952338 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.952720 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.952826 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.952836 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:09.953000 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 07:02:09.953048 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 07:02:09.953339 22460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.950006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:53055 (local address 127.13.207.3:40383)
0808 07:02:09.950068 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:09.950071 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:09.950253 (+ 182us) server_negotiation.cc:407] Connection header received
0808 07:02:09.950393 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:09.950394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:09.950407 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:09.950489 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:09.950702 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.951219 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.952401 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.952879 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.952999 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.953009 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:09.953189 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 07:02:09.953240 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:02:09.953545 22495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.949880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:40383 (local address 127.13.207.1:53055)
0808 07:02:09.950131 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:09.950152 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:09.950265 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:09.950496 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:09.950497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:09.950499 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:09.950582 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.950586 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.951325 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.951326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.952278 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.952281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.953026 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.953028 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.953105 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.953109 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:02:09.953181 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 07:02:09.953381 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":3,"thread_start_us":161,"threads_started":1}
I20260808 07:02:09.953752 22238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65839adf20604c2888e110a810386edd" candidate_uuid: "ec60aadcbf224a4aabffcd82f789dc66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef8811953fbe41e5821e087f2e58eab0" is_pre_election: true
I20260808 07:02:09.953893 22238 raft_consensus.cc:2454] T 65839adf20604c2888e110a810386edd P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec60aadcbf224a4aabffcd82f789dc66 in term 0.
I20260808 07:02:09.953877 22384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65839adf20604c2888e110a810386edd" candidate_uuid: "ec60aadcbf224a4aabffcd82f789dc66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41e555b6549a45168d2b12376f64daea" is_pre_election: true
I20260808 07:02:09.954068 22384 raft_consensus.cc:2454] T 65839adf20604c2888e110a810386edd P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec60aadcbf224a4aabffcd82f789dc66 in term 0.
I20260808 07:02:09.954241 22023 leader_election.cc:302] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec60aadcbf224a4aabffcd82f789dc66, ef8811953fbe41e5821e087f2e58eab0; no voters:
I20260808 07:02:09.954388 22470 raft_consensus.cc:2781] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:02:09.954434 22470 raft_consensus.cc:482] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:02:09.954461 22470 raft_consensus.cc:3037] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:09.955289 22470 raft_consensus.cc:504] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:09.955631 22470 leader_election.cc:288] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66 [CANDIDATE]: Term 1 election: Requested vote from peers ef8811953fbe41e5821e087f2e58eab0 (127.13.207.2:46755), 41e555b6549a45168d2b12376f64daea (127.13.207.3:40383)
I20260808 07:02:09.955928 22384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65839adf20604c2888e110a810386edd" candidate_uuid: "ec60aadcbf224a4aabffcd82f789dc66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41e555b6549a45168d2b12376f64daea"
I20260808 07:02:09.956038 22384 raft_consensus.cc:3037] T 65839adf20604c2888e110a810386edd P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:09.956043 22238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65839adf20604c2888e110a810386edd" candidate_uuid: "ec60aadcbf224a4aabffcd82f789dc66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef8811953fbe41e5821e087f2e58eab0"
I20260808 07:02:09.956249 22238 raft_consensus.cc:3037] T 65839adf20604c2888e110a810386edd P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:09.957019 22384 raft_consensus.cc:2454] T 65839adf20604c2888e110a810386edd P 41e555b6549a45168d2b12376f64daea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec60aadcbf224a4aabffcd82f789dc66 in term 1.
I20260808 07:02:09.957403 22021 leader_election.cc:302] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41e555b6549a45168d2b12376f64daea, ec60aadcbf224a4aabffcd82f789dc66; no voters:
I20260808 07:02:09.957486 22238 raft_consensus.cc:2454] T 65839adf20604c2888e110a810386edd P ef8811953fbe41e5821e087f2e58eab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec60aadcbf224a4aabffcd82f789dc66 in term 1.
I20260808 07:02:09.957609 22470 raft_consensus.cc:2781] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:02:09.958024 22470 raft_consensus.cc:686] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66 [term 1 LEADER]: Becoming Leader. State: Replica: ec60aadcbf224a4aabffcd82f789dc66, State: Running, Role: LEADER
I20260808 07:02:09.958138 22470 consensus_queue.cc:237] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:09.959337 21923 catalog_manager.cc:5495] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec60aadcbf224a4aabffcd82f789dc66 (127.13.207.1). New cstate: current_term: 1 leader_uuid: "ec60aadcbf224a4aabffcd82f789dc66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } health_report { overall_health: UNKNOWN } } }
Aug 08 07:02:09 dist-test-slave-8tdl krb5kdc[21878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172528, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.62@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM
I20260808 07:02:10.000104 22505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.991644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38865 (local address 127.0.0.1:43082)
0808 07:02:09.991920 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:09.991933 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:09.992001 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:09.992187 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:09.992188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:09.992503 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:09.992556 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.992559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.993303 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.993304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.993863 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:09.993865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.994303 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.994304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:09.994355 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.995073 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:09.995077 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:09.996932 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:09.998647 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:09.998656 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:09.998658 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:09.998990 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:09.999228 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:09.999229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:09.999229 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:09.999301 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:09.999723 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:09.999732 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:09.999793 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 07:02:09.999888 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 07:02:10.000004 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":127,"threads_started":1}
I20260808 07:02:10.000206 21978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:09.991774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43082 (local address 127.13.207.62:38865)
0808 07:02:09.991840 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:09.991847 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:09.992054 (+ 207us) server_negotiation.cc:407] Connection header received
0808 07:02:09.992084 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:09.992084 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:09.992094 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:09.992143 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:09.992659 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.993202 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.993962 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:09.994187 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:09.994260 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:09.997060 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:09.997062 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:09.997062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:09.997070 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:09.998550 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:09.999070 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:09.999071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:09.999072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:09.999133 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:09.999409 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:09.999410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:09.999410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:09.999601 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:09.999684 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:09.999987 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 07:02:10.000139 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:02:10.003288 21923 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:43082:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 07:02:10.003803 21923 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:02:10.006254 22069 tablet_service.cc:1467] Processing CreateTablet for tablet 258c4a75182e4783b337a1bdff4dea2f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a5dbc1a2237a46888ddd657415a0e6d7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:02:10.006232 22217 tablet_service.cc:1467] Processing CreateTablet for tablet 258c4a75182e4783b337a1bdff4dea2f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a5dbc1a2237a46888ddd657415a0e6d7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:02:10.006235 22364 tablet_service.cc:1467] Processing CreateTablet for tablet 258c4a75182e4783b337a1bdff4dea2f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a5dbc1a2237a46888ddd657415a0e6d7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:02:10.006475 22069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 258c4a75182e4783b337a1bdff4dea2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:10.006474 22217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 258c4a75182e4783b337a1bdff4dea2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:10.006480 22364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 258c4a75182e4783b337a1bdff4dea2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:10.008510 22463 tablet_bootstrap.cc:492] T 258c4a75182e4783b337a1bdff4dea2f P ef8811953fbe41e5821e087f2e58eab0: Bootstrap starting.
I20260808 07:02:10.008533 22464 tablet_bootstrap.cc:492] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66: Bootstrap starting.
I20260808 07:02:10.009240 22464 tablet_bootstrap.cc:654] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:10.009262 22463 tablet_bootstrap.cc:654] T 258c4a75182e4783b337a1bdff4dea2f P ef8811953fbe41e5821e087f2e58eab0: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:10.010268 22465 tablet_bootstrap.cc:492] T 258c4a75182e4783b337a1bdff4dea2f P 41e555b6549a45168d2b12376f64daea: Bootstrap starting.
I20260808 07:02:10.010344 22463 tablet_bootstrap.cc:492] T 258c4a75182e4783b337a1bdff4dea2f P ef8811953fbe41e5821e087f2e58eab0: No bootstrap required, opened a new log
I20260808 07:02:10.010406 22463 ts_tablet_manager.cc:1397] T 258c4a75182e4783b337a1bdff4dea2f P ef8811953fbe41e5821e087f2e58eab0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:02:10.010614 22463 raft_consensus.cc:348] T 258c4a75182e4783b337a1bdff4dea2f P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.010730 22463 raft_consensus.cc:374] T 258c4a75182e4783b337a1bdff4dea2f P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:10.010754 22463 raft_consensus.cc:729] T 258c4a75182e4783b337a1bdff4dea2f P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef8811953fbe41e5821e087f2e58eab0, State: Initialized, Role: FOLLOWER
I20260808 07:02:10.010742 22464 tablet_bootstrap.cc:492] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66: No bootstrap required, opened a new log
I20260808 07:02:10.010776 22465 tablet_bootstrap.cc:654] T 258c4a75182e4783b337a1bdff4dea2f P 41e555b6549a45168d2b12376f64daea: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:10.010802 22464 ts_tablet_manager.cc:1397] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:02:10.010803 22463 consensus_queue.cc:260] T 258c4a75182e4783b337a1bdff4dea2f P ef8811953fbe41e5821e087f2e58eab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.011013 22463 ts_tablet_manager.cc:1428] T 258c4a75182e4783b337a1bdff4dea2f P ef8811953fbe41e5821e087f2e58eab0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:10.011081 22464 raft_consensus.cc:348] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.011183 22464 raft_consensus.cc:374] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:10.011204 22464 raft_consensus.cc:729] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec60aadcbf224a4aabffcd82f789dc66, State: Initialized, Role: FOLLOWER
I20260808 07:02:10.011273 22464 consensus_queue.cc:260] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.011602 22464 ts_tablet_manager.cc:1428] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:10.011757 22470 tablet_replica.cc:439] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } } }
I20260808 07:02:10.011791 22469 tablet_replica.cc:439] T 258c4a75182e4783b337a1bdff4dea2f P ef8811953fbe41e5821e087f2e58eab0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } } }
I20260808 07:02:10.011934 22470 tablet_replica.cc:442] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:02:10.011986 22469 tablet_replica.cc:442] T 258c4a75182e4783b337a1bdff4dea2f P ef8811953fbe41e5821e087f2e58eab0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:02:10.013005 22465 tablet_bootstrap.cc:492] T 258c4a75182e4783b337a1bdff4dea2f P 41e555b6549a45168d2b12376f64daea: No bootstrap required, opened a new log
I20260808 07:02:10.013108 22465 ts_tablet_manager.cc:1397] T 258c4a75182e4783b337a1bdff4dea2f P 41e555b6549a45168d2b12376f64daea: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 07:02:10.013255 22465 raft_consensus.cc:348] T 258c4a75182e4783b337a1bdff4dea2f P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.013331 22465 raft_consensus.cc:374] T 258c4a75182e4783b337a1bdff4dea2f P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:10.013346 22465 raft_consensus.cc:729] T 258c4a75182e4783b337a1bdff4dea2f P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41e555b6549a45168d2b12376f64daea, State: Initialized, Role: FOLLOWER
I20260808 07:02:10.013412 22465 consensus_queue.cc:260] T 258c4a75182e4783b337a1bdff4dea2f P 41e555b6549a45168d2b12376f64daea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.013509 22465 ts_tablet_manager.cc:1428] T 258c4a75182e4783b337a1bdff4dea2f P 41e555b6549a45168d2b12376f64daea: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:10.013525 22474 tablet_replica.cc:439] T 258c4a75182e4783b337a1bdff4dea2f P 41e555b6549a45168d2b12376f64daea: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } } }
I20260808 07:02:10.013602 22474 tablet_replica.cc:442] T 258c4a75182e4783b337a1bdff4dea2f P 41e555b6549a45168d2b12376f64daea: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 07:02:10.038017 22435 tablet.cc:2369] T 258c4a75182e4783b337a1bdff4dea2f P 41e555b6549a45168d2b12376f64daea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:02:10.177973 22470 raft_consensus.cc:482] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:02:10.178099 22470 raft_consensus.cc:504] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.178295 22470 leader_election.cc:288] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef8811953fbe41e5821e087f2e58eab0 (127.13.207.2:46755), 41e555b6549a45168d2b12376f64daea (127.13.207.3:40383)
I20260808 07:02:10.178592 22238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258c4a75182e4783b337a1bdff4dea2f" candidate_uuid: "ec60aadcbf224a4aabffcd82f789dc66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef8811953fbe41e5821e087f2e58eab0" is_pre_election: true
I20260808 07:02:10.178756 22238 raft_consensus.cc:2454] T 258c4a75182e4783b337a1bdff4dea2f P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec60aadcbf224a4aabffcd82f789dc66 in term 0.
I20260808 07:02:10.178872 22384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258c4a75182e4783b337a1bdff4dea2f" candidate_uuid: "ec60aadcbf224a4aabffcd82f789dc66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41e555b6549a45168d2b12376f64daea" is_pre_election: true
I20260808 07:02:10.179046 22384 raft_consensus.cc:2454] T 258c4a75182e4783b337a1bdff4dea2f P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec60aadcbf224a4aabffcd82f789dc66 in term 0.
I20260808 07:02:10.179160 22023 leader_election.cc:302] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec60aadcbf224a4aabffcd82f789dc66, ef8811953fbe41e5821e087f2e58eab0; no voters:
I20260808 07:02:10.179279 22470 raft_consensus.cc:2781] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:02:10.179325 22470 raft_consensus.cc:482] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:02:10.179348 22470 raft_consensus.cc:3037] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:10.180099 22470 raft_consensus.cc:504] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.180286 22470 leader_election.cc:288] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66 [CANDIDATE]: Term 1 election: Requested vote from peers ef8811953fbe41e5821e087f2e58eab0 (127.13.207.2:46755), 41e555b6549a45168d2b12376f64daea (127.13.207.3:40383)
I20260808 07:02:10.180469 22238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258c4a75182e4783b337a1bdff4dea2f" candidate_uuid: "ec60aadcbf224a4aabffcd82f789dc66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef8811953fbe41e5821e087f2e58eab0"
I20260808 07:02:10.180505 22384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258c4a75182e4783b337a1bdff4dea2f" candidate_uuid: "ec60aadcbf224a4aabffcd82f789dc66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41e555b6549a45168d2b12376f64daea"
I20260808 07:02:10.180583 22384 raft_consensus.cc:3037] T 258c4a75182e4783b337a1bdff4dea2f P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:10.180569 22238 raft_consensus.cc:3037] T 258c4a75182e4783b337a1bdff4dea2f P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:10.181453 22384 raft_consensus.cc:2454] T 258c4a75182e4783b337a1bdff4dea2f P 41e555b6549a45168d2b12376f64daea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec60aadcbf224a4aabffcd82f789dc66 in term 1.
I20260808 07:02:10.181454 22238 raft_consensus.cc:2454] T 258c4a75182e4783b337a1bdff4dea2f P ef8811953fbe41e5821e087f2e58eab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec60aadcbf224a4aabffcd82f789dc66 in term 1.
I20260808 07:02:10.181701 22023 leader_election.cc:302] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec60aadcbf224a4aabffcd82f789dc66, ef8811953fbe41e5821e087f2e58eab0; no voters:
I20260808 07:02:10.181830 22470 raft_consensus.cc:2781] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:02:10.181900 22470 raft_consensus.cc:686] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66 [term 1 LEADER]: Becoming Leader. State: Replica: ec60aadcbf224a4aabffcd82f789dc66, State: Running, Role: LEADER
I20260808 07:02:10.181949 22470 consensus_queue.cc:237] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.182363 22497 tablet_replica.cc:439] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66: TxnStatusTablet state changed. Reason: New leader ec60aadcbf224a4aabffcd82f789dc66. Latest consensus state: current_term: 1 leader_uuid: "ec60aadcbf224a4aabffcd82f789dc66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } } }
I20260808 07:02:10.182487 22497 tablet_replica.cc:442] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:02:10.182710 22510 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 07:02:10.182832 22510 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 07:02:10.182825 21926 catalog_manager.cc:5495] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec60aadcbf224a4aabffcd82f789dc66 (127.13.207.1). New cstate: current_term: 1 leader_uuid: "ec60aadcbf224a4aabffcd82f789dc66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } health_report { overall_health: UNKNOWN } } }
I20260808 07:02:10.205471 22505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:10.202652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:38045 (local address 127.0.0.1:52834)
0808 07:02:10.202789 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:10.202806 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:10.202886 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:10.203118 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:10.203119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:10.203121 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:02:10.203188 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:10.203190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.203856 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.203857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:10.204445 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:10.204447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.204833 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.204834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:10.204901 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:10.204920 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:02:10.205263 (+ 343us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:02:10.205275 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 07:02:10.205328 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 07:02:10.205391 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 07:02:10.205464 22462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:10.202846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52834 (local address 127.13.207.1:38045)
0808 07:02:10.202921 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:10.202924 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:10.202947 (+ 23us) server_negotiation.cc:407] Connection header received
0808 07:02:10.203000 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:10.203001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:10.203021 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:10.203096 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:02:10.203270 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.203768 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.204543 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.204750 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.204831 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:10.205042 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:02:10.205188 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:02:10.205273 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:10.205331 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 07:02:10.205354 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 07:02:10.207435 22238 raft_consensus.cc:1261] T 258c4a75182e4783b337a1bdff4dea2f P ef8811953fbe41e5821e087f2e58eab0 [term 1 FOLLOWER]: Refusing update from remote peer ec60aadcbf224a4aabffcd82f789dc66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:10.207610 22384 raft_consensus.cc:1261] T 258c4a75182e4783b337a1bdff4dea2f P 41e555b6549a45168d2b12376f64daea [term 1 FOLLOWER]: Refusing update from remote peer ec60aadcbf224a4aabffcd82f789dc66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:10.207788 22470 consensus_queue.cc:1035] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:10.207937 22497 consensus_queue.cc:1035] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:10.208772 22469 tablet_replica.cc:439] T 258c4a75182e4783b337a1bdff4dea2f P ef8811953fbe41e5821e087f2e58eab0: TxnStatusTablet state changed. Reason: New leader ec60aadcbf224a4aabffcd82f789dc66. Latest consensus state: current_term: 1 leader_uuid: "ec60aadcbf224a4aabffcd82f789dc66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } } }
I20260808 07:02:10.208891 22469 tablet_replica.cc:442] T 258c4a75182e4783b337a1bdff4dea2f P ef8811953fbe41e5821e087f2e58eab0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:02:10.209313 22474 tablet_replica.cc:439] T 258c4a75182e4783b337a1bdff4dea2f P 41e555b6549a45168d2b12376f64daea: TxnStatusTablet state changed. Reason: New leader ec60aadcbf224a4aabffcd82f789dc66. Latest consensus state: current_term: 1 leader_uuid: "ec60aadcbf224a4aabffcd82f789dc66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } } }
I20260808 07:02:10.209424 22474 tablet_replica.cc:442] T 258c4a75182e4783b337a1bdff4dea2f P 41e555b6549a45168d2b12376f64daea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:02:10.209512 22470 tablet_replica.cc:439] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec60aadcbf224a4aabffcd82f789dc66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } } }
I20260808 07:02:10.209604 22470 tablet_replica.cc:442] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:02:10.209939 22469 tablet_replica.cc:439] T 258c4a75182e4783b337a1bdff4dea2f P ef8811953fbe41e5821e087f2e58eab0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec60aadcbf224a4aabffcd82f789dc66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } } }
I20260808 07:02:10.210039 22469 tablet_replica.cc:442] T 258c4a75182e4783b337a1bdff4dea2f P ef8811953fbe41e5821e087f2e58eab0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:02:10.210944 22497 tablet_replica.cc:439] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec60aadcbf224a4aabffcd82f789dc66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } } }
I20260808 07:02:10.211104 22497 tablet_replica.cc:442] T 258c4a75182e4783b337a1bdff4dea2f P ec60aadcbf224a4aabffcd82f789dc66: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:02:10.212062 22477 mvcc.cc:205] Tried to move back new op lower bound from 7316162683726733312 to 7316162683625844736. Current Snapshot: MvccSnapshot[applied={T|T < 7316162683726733312}]
I20260808 07:02:10.212217 22474 tablet_replica.cc:439] T 258c4a75182e4783b337a1bdff4dea2f P 41e555b6549a45168d2b12376f64daea: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec60aadcbf224a4aabffcd82f789dc66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } } }
I20260808 07:02:10.212301 22474 tablet_replica.cc:442] T 258c4a75182e4783b337a1bdff4dea2f P 41e555b6549a45168d2b12376f64daea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:02:10.216024 22462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:10.212546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52840 (local address 127.13.207.1:38045)
0808 07:02:10.212619 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:10.212621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:10.212629 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:02:10.212652 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:10.212652 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:10.212663 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:10.212709 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:02:10.212875 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.214028 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.215059 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.215370 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.215449 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:10.215697 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:02:10.215736 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:02:10.215794 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:10.215900 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 07:02:10.215926 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 07:02:10.224471 22462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:10.221208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:34939 (local address 127.13.207.1:38045)
0808 07:02:10.221278 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:10.221281 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:10.221292 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:02:10.221347 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:10.221348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:10.221375 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:10.221443 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:02:10.221652 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.222500 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.223447 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.223751 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.223828 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:10.224104 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:02:10.224141 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:02:10.224202 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:10.224328 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 07:02:10.224358 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":46}
I20260808 07:02:10.224475 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:10.220848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:38045 (local address 127.13.207.1:34939)
0808 07:02:10.221212 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:10.221226 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:10.221297 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:10.221491 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:10.221493 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:10.221495 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:02:10.221566 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:10.221568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.222603 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.222604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:10.223353 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:10.223355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.223885 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.223886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:10.223940 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:10.223994 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:02:10.224246 (+ 252us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:02:10.224249 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:02:10.224308 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 07:02:10.224359 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":2,"thread_start_us":82,"threads_started":1}
I20260808 07:02:10.227629 22384 raft_consensus.cc:1261] T 65839adf20604c2888e110a810386edd P 41e555b6549a45168d2b12376f64daea [term 1 FOLLOWER]: Refusing update from remote peer ec60aadcbf224a4aabffcd82f789dc66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:10.227648 22238 raft_consensus.cc:1261] T 65839adf20604c2888e110a810386edd P ef8811953fbe41e5821e087f2e58eab0 [term 1 FOLLOWER]: Refusing update from remote peer ec60aadcbf224a4aabffcd82f789dc66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:10.227862 22497 consensus_queue.cc:1035] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:10.228015 22497 consensus_queue.cc:1035] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:10.246539 21926 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:02:10.246731 21926 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:02:10.247707 21926 catalog_manager.cc:5782] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664: Sending DeleteTablet for 3 replicas of tablet 65839adf20604c2888e110a810386edd
I20260808 07:02:10.248118 22364 tablet_service.cc:1514] Processing DeleteTablet for tablet 65839adf20604c2888e110a810386edd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:10 UTC) from {username='oryx', principal='oryx/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:60512
I20260808 07:02:10.248211 22069 tablet_service.cc:1514] Processing DeleteTablet for tablet 65839adf20604c2888e110a810386edd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:10 UTC) from {username='oryx', principal='oryx/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:52822
I20260808 07:02:10.248225 22217 tablet_service.cc:1514] Processing DeleteTablet for tablet 65839adf20604c2888e110a810386edd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:10 UTC) from {username='oryx', principal='oryx/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:33892
I20260808 07:02:10.248353 22491 tablet_replica.cc:330] T 65839adf20604c2888e110a810386edd P 41e555b6549a45168d2b12376f64daea: stopping tablet replica
I20260808 07:02:10.248476 22491 raft_consensus.cc:2229] T 65839adf20604c2888e110a810386edd P 41e555b6549a45168d2b12376f64daea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:02:10.248479 22489 tablet_replica.cc:330] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66: stopping tablet replica
I20260808 07:02:10.248567 22491 raft_consensus.cc:2258] T 65839adf20604c2888e110a810386edd P 41e555b6549a45168d2b12376f64daea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:10.248584 22489 raft_consensus.cc:2229] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:02:10.248759 22489 raft_consensus.cc:2258] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:10.248891 22490 tablet_replica.cc:330] T 65839adf20604c2888e110a810386edd P ef8811953fbe41e5821e087f2e58eab0: stopping tablet replica
I20260808 07:02:10.248977 22490 raft_consensus.cc:2229] T 65839adf20604c2888e110a810386edd P ef8811953fbe41e5821e087f2e58eab0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:02:10.249032 22490 raft_consensus.cc:2258] T 65839adf20604c2888e110a810386edd P ef8811953fbe41e5821e087f2e58eab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:10.249536 22490 ts_tablet_manager.cc:1905] T 65839adf20604c2888e110a810386edd P ef8811953fbe41e5821e087f2e58eab0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:10.249739 22489 ts_tablet_manager.cc:1905] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:10.251968 22491 ts_tablet_manager.cc:1905] T 65839adf20604c2888e110a810386edd P 41e555b6549a45168d2b12376f64daea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:10.252547 22490 ts_tablet_manager.cc:1918] T 65839adf20604c2888e110a810386edd P ef8811953fbe41e5821e087f2e58eab0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:02:10.252627 22490 log.cc:1196] T 65839adf20604c2888e110a810386edd P ef8811953fbe41e5821e087f2e58eab0: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-1/wal/wals/65839adf20604c2888e110a810386edd
I20260808 07:02:10.252874 22490 ts_tablet_manager.cc:1939] T 65839adf20604c2888e110a810386edd P ef8811953fbe41e5821e087f2e58eab0: Deleting consensus metadata
I20260808 07:02:10.253857 22489 ts_tablet_manager.cc:1918] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:02:10.253878 22491 ts_tablet_manager.cc:1918] T 65839adf20604c2888e110a810386edd P 41e555b6549a45168d2b12376f64daea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:02:10.253927 22491 log.cc:1196] T 65839adf20604c2888e110a810386edd P 41e555b6549a45168d2b12376f64daea: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/65839adf20604c2888e110a810386edd
I20260808 07:02:10.253927 22489 log.cc:1196] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-0/wal/wals/65839adf20604c2888e110a810386edd
I20260808 07:02:10.254154 22491 ts_tablet_manager.cc:1939] T 65839adf20604c2888e110a810386edd P 41e555b6549a45168d2b12376f64daea: Deleting consensus metadata
I20260808 07:02:10.254177 22489 ts_tablet_manager.cc:1939] T 65839adf20604c2888e110a810386edd P ec60aadcbf224a4aabffcd82f789dc66: Deleting consensus metadata
I20260808 07:02:10.254621 21911 catalog_manager.cc:4841] TS ec60aadcbf224a4aabffcd82f789dc66 (127.13.207.1:38045): tablet 65839adf20604c2888e110a810386edd (table test-table [id=ebb1abea234448c7b66d32f77f1615c0]) successfully deleted
I20260808 07:02:10.254753 21911 catalog_manager.cc:4841] TS ef8811953fbe41e5821e087f2e58eab0 (127.13.207.2:46755): tablet 65839adf20604c2888e110a810386edd (table test-table [id=ebb1abea234448c7b66d32f77f1615c0]) successfully deleted
I20260808 07:02:10.254796 21909 catalog_manager.cc:4841] TS 41e555b6549a45168d2b12376f64daea (127.13.207.3:40383): tablet 65839adf20604c2888e110a810386edd (table test-table [id=ebb1abea234448c7b66d32f77f1615c0]) successfully deleted
Aug 08 07:02:10 dist-test-slave-8tdl krb5kdc[21878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:02:10 dist-test-slave-8tdl krb5kdc[21878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:10 dist-test-slave-8tdl krb5kdc[21878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
W20260808 07:02:10.286949 21978 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:10.279377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43086 (local address 127.13.207.62:38865)
0808 07:02:10.279458 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:10.279461 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:10.279597 (+ 136us) server_negotiation.cc:407] Connection header received
0808 07:02:10.279712 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:10.279713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:10.279727 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:10.279814 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:10.280658 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.281529 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.282448 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.282766 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.282854 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:10.286856 (+ 4002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43086: BlockingRecv error: recv got EOF from 127.0.0.1:43086 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 07:02:10 dist-test-slave-8tdl krb5kdc[21878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM
I20260808 07:02:10.297518 21978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:10.288142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43092 (local address 127.13.207.62:38865)
0808 07:02:10.288217 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:10.288219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:10.288227 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:02:10.288253 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:10.288254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:10.288267 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:10.288340 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:10.288878 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.289389 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.290333 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.290577 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.290665 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:10.293796 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:10.293798 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:10.293798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:10.293812 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:10.295779 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:10.296299 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:10.296300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:10.296300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:10.296369 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:10.296645 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:10.296646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:10.296646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:10.296795 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:10.296901 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:10.297160 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 07:02:10.297253 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 07:02:10.300263 21926 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:02:10.300560 21926 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:02:10.302347 22217 tablet_service.cc:1467] Processing CreateTablet for tablet 3d1a55e2c6724715a0c01cd1d88bfea8 (DEFAULT_TABLE table=test-table [id=012e89290b104a9d9134234e5a4d3659]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:10.302366 22069 tablet_service.cc:1467] Processing CreateTablet for tablet 3d1a55e2c6724715a0c01cd1d88bfea8 (DEFAULT_TABLE table=test-table [id=012e89290b104a9d9134234e5a4d3659]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:10.302358 22364 tablet_service.cc:1467] Processing CreateTablet for tablet 3d1a55e2c6724715a0c01cd1d88bfea8 (DEFAULT_TABLE table=test-table [id=012e89290b104a9d9134234e5a4d3659]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:10.302524 22069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d1a55e2c6724715a0c01cd1d88bfea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:10.302523 22364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d1a55e2c6724715a0c01cd1d88bfea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:10.302524 22217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d1a55e2c6724715a0c01cd1d88bfea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:10.304082 22463 tablet_bootstrap.cc:492] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0: Bootstrap starting.
I20260808 07:02:10.304087 22464 tablet_bootstrap.cc:492] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ec60aadcbf224a4aabffcd82f789dc66: Bootstrap starting.
I20260808 07:02:10.304101 22465 tablet_bootstrap.cc:492] T 3d1a55e2c6724715a0c01cd1d88bfea8 P 41e555b6549a45168d2b12376f64daea: Bootstrap starting.
I20260808 07:02:10.304538 22463 tablet_bootstrap.cc:654] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:10.304734 22464 tablet_bootstrap.cc:654] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ec60aadcbf224a4aabffcd82f789dc66: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:10.304734 22465 tablet_bootstrap.cc:654] T 3d1a55e2c6724715a0c01cd1d88bfea8 P 41e555b6549a45168d2b12376f64daea: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:10.305502 22465 tablet_bootstrap.cc:492] T 3d1a55e2c6724715a0c01cd1d88bfea8 P 41e555b6549a45168d2b12376f64daea: No bootstrap required, opened a new log
I20260808 07:02:10.305579 22465 ts_tablet_manager.cc:1397] T 3d1a55e2c6724715a0c01cd1d88bfea8 P 41e555b6549a45168d2b12376f64daea: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:02:10.305755 22465 raft_consensus.cc:348] T 3d1a55e2c6724715a0c01cd1d88bfea8 P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.305857 22465 raft_consensus.cc:374] T 3d1a55e2c6724715a0c01cd1d88bfea8 P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:10.305887 22465 raft_consensus.cc:729] T 3d1a55e2c6724715a0c01cd1d88bfea8 P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41e555b6549a45168d2b12376f64daea, State: Initialized, Role: FOLLOWER
I20260808 07:02:10.305943 22465 consensus_queue.cc:260] T 3d1a55e2c6724715a0c01cd1d88bfea8 P 41e555b6549a45168d2b12376f64daea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.306023 22463 tablet_bootstrap.cc:492] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0: No bootstrap required, opened a new log
I20260808 07:02:10.306039 22465 ts_tablet_manager.cc:1428] T 3d1a55e2c6724715a0c01cd1d88bfea8 P 41e555b6549a45168d2b12376f64daea: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:02:10.306082 22463 ts_tablet_manager.cc:1397] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:02:10.306238 22463 raft_consensus.cc:348] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.306313 22463 raft_consensus.cc:374] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:10.306334 22463 raft_consensus.cc:729] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef8811953fbe41e5821e087f2e58eab0, State: Initialized, Role: FOLLOWER
I20260808 07:02:10.306372 22463 consensus_queue.cc:260] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.306449 22463 ts_tablet_manager.cc:1428] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:10.306746 22464 tablet_bootstrap.cc:492] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ec60aadcbf224a4aabffcd82f789dc66: No bootstrap required, opened a new log
I20260808 07:02:10.306802 22464 ts_tablet_manager.cc:1397] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ec60aadcbf224a4aabffcd82f789dc66: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 07:02:10.307013 22464 raft_consensus.cc:348] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.307097 22464 raft_consensus.cc:374] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:10.307120 22464 raft_consensus.cc:729] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec60aadcbf224a4aabffcd82f789dc66, State: Initialized, Role: FOLLOWER
I20260808 07:02:10.307173 22464 consensus_queue.cc:260] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ec60aadcbf224a4aabffcd82f789dc66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.307289 22464 ts_tablet_manager.cc:1428] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ec60aadcbf224a4aabffcd82f789dc66: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:10.360132 22469 raft_consensus.cc:482] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:02:10.360301 22469 raft_consensus.cc:504] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.360714 22469 leader_election.cc:288] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec60aadcbf224a4aabffcd82f789dc66 (127.13.207.1:38045), 41e555b6549a45168d2b12376f64daea (127.13.207.3:40383)
I20260808 07:02:10.364535 22540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:10.361001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:40383 (local address 127.13.207.2:44745)
0808 07:02:10.361227 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:10.361237 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:10.361309 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:10.361466 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:10.361467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:10.361469 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:10.361523 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:10.361526 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.362327 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.362329 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:10.363395 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:10.363397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.364202 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.364203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:10.364275 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:10.364280 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:02:10.364379 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 07:02:10.364426 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":3,"thread_start_us":140,"threads_started":1}
I20260808 07:02:10.364583 22460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:10.361106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:44745 (local address 127.13.207.3:40383)
0808 07:02:10.361161 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:10.361163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:10.361317 (+ 154us) server_negotiation.cc:407] Connection header received
0808 07:02:10.361367 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:10.361368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:10.361379 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:10.361450 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:10.361620 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.362206 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.363499 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.364072 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.364151 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:10.364161 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:10.364422 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 07:02:10.364481 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:02:10.365267 22384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d1a55e2c6724715a0c01cd1d88bfea8" candidate_uuid: "ef8811953fbe41e5821e087f2e58eab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41e555b6549a45168d2b12376f64daea" is_pre_election: true
I20260808 07:02:10.365391 22384 raft_consensus.cc:2454] T 3d1a55e2c6724715a0c01cd1d88bfea8 P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef8811953fbe41e5821e087f2e58eab0 in term 0.
I20260808 07:02:10.365589 22462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:10.361082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:52931 (local address 127.13.207.1:38045)
0808 07:02:10.361131 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:10.361133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:10.361493 (+ 360us) server_negotiation.cc:407] Connection header received
0808 07:02:10.361522 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:10.361523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:10.361532 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:10.361581 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:10.362536 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.363250 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.364571 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.365179 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.365252 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:10.365261 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:10.365456 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 07:02:10.365508 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:02:10.365602 22166 leader_election.cc:302] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41e555b6549a45168d2b12376f64daea, ef8811953fbe41e5821e087f2e58eab0; no voters:
I20260808 07:02:10.365762 22469 raft_consensus.cc:2781] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:02:10.365828 22469 raft_consensus.cc:482] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:02:10.365867 22469 raft_consensus.cc:3037] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:10.365942 22541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:10.360911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:38045 (local address 127.13.207.2:52931)
0808 07:02:10.361216 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:10.361228 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:10.361309 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:10.362352 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:10.362353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:10.362356 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:10.362438 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:10.362440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.363375 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.363376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:10.364466 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:10.364469 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.365256 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.365257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:10.365343 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:10.365346 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:02:10.365718 (+ 372us) client_negotiation.cc:240] Negotiation successful
0808 07:02:10.365771 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":74,"threads_started":1}
I20260808 07:02:10.366185 22089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d1a55e2c6724715a0c01cd1d88bfea8" candidate_uuid: "ef8811953fbe41e5821e087f2e58eab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec60aadcbf224a4aabffcd82f789dc66" is_pre_election: true
I20260808 07:02:10.366286 22089 raft_consensus.cc:2454] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef8811953fbe41e5821e087f2e58eab0 in term 0.
I20260808 07:02:10.366981 22469 raft_consensus.cc:504] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.367149 22469 leader_election.cc:288] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0 [CANDIDATE]: Term 1 election: Requested vote from peers ec60aadcbf224a4aabffcd82f789dc66 (127.13.207.1:38045), 41e555b6549a45168d2b12376f64daea (127.13.207.3:40383)
I20260808 07:02:10.367393 22384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d1a55e2c6724715a0c01cd1d88bfea8" candidate_uuid: "ef8811953fbe41e5821e087f2e58eab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41e555b6549a45168d2b12376f64daea"
I20260808 07:02:10.367501 22384 raft_consensus.cc:3037] T 3d1a55e2c6724715a0c01cd1d88bfea8 P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:10.367484 22089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d1a55e2c6724715a0c01cd1d88bfea8" candidate_uuid: "ef8811953fbe41e5821e087f2e58eab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec60aadcbf224a4aabffcd82f789dc66"
I20260808 07:02:10.367599 22089 raft_consensus.cc:3037] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:10.368305 22089 raft_consensus.cc:2454] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ec60aadcbf224a4aabffcd82f789dc66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef8811953fbe41e5821e087f2e58eab0 in term 1.
I20260808 07:02:10.368398 22384 raft_consensus.cc:2454] T 3d1a55e2c6724715a0c01cd1d88bfea8 P 41e555b6549a45168d2b12376f64daea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef8811953fbe41e5821e087f2e58eab0 in term 1.
I20260808 07:02:10.368512 22168 leader_election.cc:302] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ec60aadcbf224a4aabffcd82f789dc66, ef8811953fbe41e5821e087f2e58eab0; no voters:
I20260808 07:02:10.368597 22469 raft_consensus.cc:2781] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:02:10.368888 22469 raft_consensus.cc:686] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0 [term 1 LEADER]: Becoming Leader. State: Replica: ef8811953fbe41e5821e087f2e58eab0, State: Running, Role: LEADER
I20260808 07:02:10.369077 22469 consensus_queue.cc:237] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.370007 21923 catalog_manager.cc:5495] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef8811953fbe41e5821e087f2e58eab0 (127.13.207.2). New cstate: current_term: 1 leader_uuid: "ef8811953fbe41e5821e087f2e58eab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } health_report { overall_health: UNKNOWN } } }
I20260808 07:02:10.384714 22459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:10.381893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33902 (local address 127.13.207.2:46755)
0808 07:02:10.381951 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:10.381954 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:10.381964 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:02:10.382109 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:10.382110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:10.382131 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:10.382218 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:02:10.382361 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.382944 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.383754 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.383951 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.384072 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:10.384297 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:02:10.384414 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:02:10.384497 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:10.384549 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 07:02:10.384594 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:02:10.385912 22384 raft_consensus.cc:1261] T 3d1a55e2c6724715a0c01cd1d88bfea8 P 41e555b6549a45168d2b12376f64daea [term 1 FOLLOWER]: Refusing update from remote peer ef8811953fbe41e5821e087f2e58eab0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:10.385927 22089 raft_consensus.cc:1261] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ec60aadcbf224a4aabffcd82f789dc66 [term 1 FOLLOWER]: Refusing update from remote peer ef8811953fbe41e5821e087f2e58eab0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:10.386335 22469 consensus_queue.cc:1035] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:10.386643 22542 consensus_queue.cc:1035] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:10.392210 21923 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:02:10.392341 21923 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:02:10.393244 21923 catalog_manager.cc:5782] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664: Sending DeleteTablet for 3 replicas of tablet 3d1a55e2c6724715a0c01cd1d88bfea8
I20260808 07:02:10.393671 22069 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d1a55e2c6724715a0c01cd1d88bfea8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:10 UTC) from {username='oryx', principal='oryx/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:52822
I20260808 07:02:10.393721 22364 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d1a55e2c6724715a0c01cd1d88bfea8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:10 UTC) from {username='oryx', principal='oryx/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:60512
I20260808 07:02:10.393790 22489 tablet_replica.cc:330] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ec60aadcbf224a4aabffcd82f789dc66: stopping tablet replica
I20260808 07:02:10.393821 22491 tablet_replica.cc:330] T 3d1a55e2c6724715a0c01cd1d88bfea8 P 41e555b6549a45168d2b12376f64daea: stopping tablet replica
I20260808 07:02:10.393852 22489 raft_consensus.cc:2229] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ec60aadcbf224a4aabffcd82f789dc66 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:02:10.393875 22491 raft_consensus.cc:2229] T 3d1a55e2c6724715a0c01cd1d88bfea8 P 41e555b6549a45168d2b12376f64daea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:02:10.393903 22489 raft_consensus.cc:2258] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ec60aadcbf224a4aabffcd82f789dc66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:10.393918 22491 raft_consensus.cc:2258] T 3d1a55e2c6724715a0c01cd1d88bfea8 P 41e555b6549a45168d2b12376f64daea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:10.394265 22491 ts_tablet_manager.cc:1905] T 3d1a55e2c6724715a0c01cd1d88bfea8 P 41e555b6549a45168d2b12376f64daea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:10.394881 22217 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d1a55e2c6724715a0c01cd1d88bfea8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:10 UTC) from {username='oryx', principal='oryx/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:33892
I20260808 07:02:10.394999 22490 tablet_replica.cc:330] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0: stopping tablet replica
I20260808 07:02:10.395053 22490 raft_consensus.cc:2229] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:02:10.395154 22490 raft_consensus.cc:2258] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:10.395577 22490 ts_tablet_manager.cc:1905] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:10.397428 21923 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:02:10.397681 21923 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:02:10.397992 22490 ts_tablet_manager.cc:1918] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:02:10.398064 22490 log.cc:1196] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-1/wal/wals/3d1a55e2c6724715a0c01cd1d88bfea8
I20260808 07:02:10.398329 22490 ts_tablet_manager.cc:1939] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ef8811953fbe41e5821e087f2e58eab0: Deleting consensus metadata
I20260808 07:02:10.399199 21911 catalog_manager.cc:4841] TS ef8811953fbe41e5821e087f2e58eab0 (127.13.207.2:46755): tablet 3d1a55e2c6724715a0c01cd1d88bfea8 (table test-table [id=012e89290b104a9d9134234e5a4d3659]) successfully deleted
I20260808 07:02:10.399253 22491 ts_tablet_manager.cc:1918] T 3d1a55e2c6724715a0c01cd1d88bfea8 P 41e555b6549a45168d2b12376f64daea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:02:10.399336 22491 log.cc:1196] T 3d1a55e2c6724715a0c01cd1d88bfea8 P 41e555b6549a45168d2b12376f64daea: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/3d1a55e2c6724715a0c01cd1d88bfea8
I20260808 07:02:10.399606 22491 ts_tablet_manager.cc:1939] T 3d1a55e2c6724715a0c01cd1d88bfea8 P 41e555b6549a45168d2b12376f64daea: Deleting consensus metadata
I20260808 07:02:10.400027 21909 catalog_manager.cc:4841] TS 41e555b6549a45168d2b12376f64daea (127.13.207.3:40383): tablet 3d1a55e2c6724715a0c01cd1d88bfea8 (table test-table [id=012e89290b104a9d9134234e5a4d3659]) successfully deleted
I20260808 07:02:10.401108 22069 tablet_service.cc:1467] Processing CreateTablet for tablet 4d3acae5d8df4eac8ebb3814f1e9fd13 (DEFAULT_TABLE table=test-table [id=b9adb1529e424391abea5b28d07f3bfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:10.401271 22069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d3acae5d8df4eac8ebb3814f1e9fd13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:10.402762 22464 tablet_bootstrap.cc:492] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ec60aadcbf224a4aabffcd82f789dc66: Bootstrap starting.
I20260808 07:02:10.403118 22464 tablet_bootstrap.cc:654] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ec60aadcbf224a4aabffcd82f789dc66: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:10.403690 22217 tablet_service.cc:1467] Processing CreateTablet for tablet 4d3acae5d8df4eac8ebb3814f1e9fd13 (DEFAULT_TABLE table=test-table [id=b9adb1529e424391abea5b28d07f3bfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:10.403872 22217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d3acae5d8df4eac8ebb3814f1e9fd13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:10.401101 22489 ts_tablet_manager.cc:1905] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ec60aadcbf224a4aabffcd82f789dc66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:10.406888 22464 tablet_bootstrap.cc:492] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ec60aadcbf224a4aabffcd82f789dc66: No bootstrap required, opened a new log
I20260808 07:02:10.406950 22464 ts_tablet_manager.cc:1397] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ec60aadcbf224a4aabffcd82f789dc66: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 07:02:10.407075 22464 raft_consensus.cc:348] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.407145 22464 raft_consensus.cc:374] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:10.407166 22464 raft_consensus.cc:729] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec60aadcbf224a4aabffcd82f789dc66, State: Initialized, Role: FOLLOWER
I20260808 07:02:10.407212 22464 consensus_queue.cc:260] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ec60aadcbf224a4aabffcd82f789dc66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.407299 22464 ts_tablet_manager.cc:1428] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ec60aadcbf224a4aabffcd82f789dc66: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:10.408053 22489 ts_tablet_manager.cc:1918] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ec60aadcbf224a4aabffcd82f789dc66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:02:10.408113 22489 log.cc:1196] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ec60aadcbf224a4aabffcd82f789dc66: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-0/wal/wals/3d1a55e2c6724715a0c01cd1d88bfea8
I20260808 07:02:10.408525 22463 tablet_bootstrap.cc:492] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ef8811953fbe41e5821e087f2e58eab0: Bootstrap starting.
I20260808 07:02:10.408696 22364 tablet_service.cc:1467] Processing CreateTablet for tablet 4d3acae5d8df4eac8ebb3814f1e9fd13 (DEFAULT_TABLE table=test-table [id=b9adb1529e424391abea5b28d07f3bfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:10.408836 22364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d3acae5d8df4eac8ebb3814f1e9fd13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:10.409039 22463 tablet_bootstrap.cc:654] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ef8811953fbe41e5821e087f2e58eab0: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:10.408474 22489 ts_tablet_manager.cc:1939] T 3d1a55e2c6724715a0c01cd1d88bfea8 P ec60aadcbf224a4aabffcd82f789dc66: Deleting consensus metadata
I20260808 07:02:10.410463 22465 tablet_bootstrap.cc:492] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea: Bootstrap starting.
I20260808 07:02:10.410878 22465 tablet_bootstrap.cc:654] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:10.411031 22463 tablet_bootstrap.cc:492] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ef8811953fbe41e5821e087f2e58eab0: No bootstrap required, opened a new log
I20260808 07:02:10.411085 22463 ts_tablet_manager.cc:1397] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ef8811953fbe41e5821e087f2e58eab0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 07:02:10.411257 22463 raft_consensus.cc:348] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.411314 21911 catalog_manager.cc:4841] TS ec60aadcbf224a4aabffcd82f789dc66 (127.13.207.1:38045): tablet 3d1a55e2c6724715a0c01cd1d88bfea8 (table test-table [id=012e89290b104a9d9134234e5a4d3659]) successfully deleted
I20260808 07:02:10.411355 22463 raft_consensus.cc:374] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:10.411379 22463 raft_consensus.cc:729] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef8811953fbe41e5821e087f2e58eab0, State: Initialized, Role: FOLLOWER
I20260808 07:02:10.411432 22463 consensus_queue.cc:260] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ef8811953fbe41e5821e087f2e58eab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.411561 22463 ts_tablet_manager.cc:1428] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ef8811953fbe41e5821e087f2e58eab0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:10.412052 22465 tablet_bootstrap.cc:492] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea: No bootstrap required, opened a new log
I20260808 07:02:10.412190 22465 ts_tablet_manager.cc:1397] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:02:10.412495 22465 raft_consensus.cc:348] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.412628 22465 raft_consensus.cc:374] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:10.412650 22465 raft_consensus.cc:729] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41e555b6549a45168d2b12376f64daea, State: Initialized, Role: FOLLOWER
I20260808 07:02:10.412693 22465 consensus_queue.cc:260] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.412791 22465 ts_tablet_manager.cc:1428] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:02:10.600602 22474 raft_consensus.cc:482] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:02:10.600742 22474 raft_consensus.cc:504] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.600950 22474 leader_election.cc:288] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec60aadcbf224a4aabffcd82f789dc66 (127.13.207.1:38045), ef8811953fbe41e5821e087f2e58eab0 (127.13.207.2:46755)
I20260808 07:02:10.601233 22089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3acae5d8df4eac8ebb3814f1e9fd13" candidate_uuid: "41e555b6549a45168d2b12376f64daea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec60aadcbf224a4aabffcd82f789dc66" is_pre_election: true
I20260808 07:02:10.601285 22238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3acae5d8df4eac8ebb3814f1e9fd13" candidate_uuid: "41e555b6549a45168d2b12376f64daea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef8811953fbe41e5821e087f2e58eab0" is_pre_election: true
I20260808 07:02:10.601375 22089 raft_consensus.cc:2454] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41e555b6549a45168d2b12376f64daea in term 0.
I20260808 07:02:10.601401 22238 raft_consensus.cc:2454] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41e555b6549a45168d2b12376f64daea in term 0.
I20260808 07:02:10.601632 22317 leader_election.cc:302] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41e555b6549a45168d2b12376f64daea, ec60aadcbf224a4aabffcd82f789dc66; no voters:
I20260808 07:02:10.601828 22474 raft_consensus.cc:2781] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:02:10.601897 22474 raft_consensus.cc:482] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:02:10.601928 22474 raft_consensus.cc:3037] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:10.603163 22474 raft_consensus.cc:504] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.603350 22474 leader_election.cc:288] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea [CANDIDATE]: Term 1 election: Requested vote from peers ec60aadcbf224a4aabffcd82f789dc66 (127.13.207.1:38045), ef8811953fbe41e5821e087f2e58eab0 (127.13.207.2:46755)
I20260808 07:02:10.603585 22089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3acae5d8df4eac8ebb3814f1e9fd13" candidate_uuid: "41e555b6549a45168d2b12376f64daea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec60aadcbf224a4aabffcd82f789dc66"
I20260808 07:02:10.603689 22089 raft_consensus.cc:3037] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ec60aadcbf224a4aabffcd82f789dc66 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:10.603637 22238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3acae5d8df4eac8ebb3814f1e9fd13" candidate_uuid: "41e555b6549a45168d2b12376f64daea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef8811953fbe41e5821e087f2e58eab0"
I20260808 07:02:10.603734 22238 raft_consensus.cc:3037] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ef8811953fbe41e5821e087f2e58eab0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:10.604583 22089 raft_consensus.cc:2454] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ec60aadcbf224a4aabffcd82f789dc66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41e555b6549a45168d2b12376f64daea in term 1.
I20260808 07:02:10.604709 22238 raft_consensus.cc:2454] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ef8811953fbe41e5821e087f2e58eab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41e555b6549a45168d2b12376f64daea in term 1.
I20260808 07:02:10.604820 22317 leader_election.cc:302] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41e555b6549a45168d2b12376f64daea, ec60aadcbf224a4aabffcd82f789dc66; no voters:
I20260808 07:02:10.604938 22474 raft_consensus.cc:2781] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:02:10.605082 22474 raft_consensus.cc:686] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea [term 1 LEADER]: Becoming Leader. State: Replica: 41e555b6549a45168d2b12376f64daea, State: Running, Role: LEADER
I20260808 07:02:10.605159 22474 consensus_queue.cc:237] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } }
I20260808 07:02:10.605873 21923 catalog_manager.cc:5495] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea reported cstate change: term changed from 0 to 1, leader changed from <none> to 41e555b6549a45168d2b12376f64daea (127.13.207.3). New cstate: current_term: 1 leader_uuid: "41e555b6549a45168d2b12376f64daea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41e555b6549a45168d2b12376f64daea" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40383 } health_report { overall_health: HEALTHY } } }
I20260808 07:02:10.655521 22460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:10.652437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60534 (local address 127.13.207.3:40383)
0808 07:02:10.652532 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:10.652534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:10.652545 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:02:10.652606 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:10.652607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:10.652620 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:10.652696 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:02:10.652811 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.653453 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.654637 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.654899 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.654997 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:10.655177 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:02:10.655223 (+ 46us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:02:10.655294 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:10.655384 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 07:02:10.655411 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 07:02:10.661935 22556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:10.659130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:38045 (local address 127.13.207.3:33887)
0808 07:02:10.659483 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:10.659502 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:10.659605 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:10.659734 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:10.659735 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:10.659737 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:02:10.659792 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:10.659794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.660475 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.660475 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:10.661070 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:10.661072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.661468 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.661469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:10.661533 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:10.661551 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:02:10.661752 (+ 201us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:02:10.661755 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:02:10.661799 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 07:02:10.661858 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":130,"threads_started":1}
I20260808 07:02:10.661929 22462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:10.659254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:33887 (local address 127.13.207.1:38045)
0808 07:02:10.659443 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:10.659446 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:10.659591 (+ 145us) server_negotiation.cc:407] Connection header received
0808 07:02:10.659667 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:10.659668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:10.659679 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:10.659732 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:02:10.659868 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.660380 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.661171 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.661381 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.661461 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:10.661642 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:02:10.661686 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:02:10.661750 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:10.661815 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 07:02:10.661836 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260808 07:02:10.664986 22089 raft_consensus.cc:1261] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ec60aadcbf224a4aabffcd82f789dc66 [term 1 FOLLOWER]: Refusing update from remote peer 41e555b6549a45168d2b12376f64daea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:10.665001 22238 raft_consensus.cc:1261] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ef8811953fbe41e5821e087f2e58eab0 [term 1 FOLLOWER]: Refusing update from remote peer 41e555b6549a45168d2b12376f64daea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:10.665233 22474 consensus_queue.cc:1035] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef8811953fbe41e5821e087f2e58eab0" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 46755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:10.665361 22552 consensus_queue.cc:1035] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec60aadcbf224a4aabffcd82f789dc66" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:10.673512 22460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:10.670605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:52975 (local address 127.13.207.3:40383)
0808 07:02:10.670655 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:10.670657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:10.670708 (+ 51us) server_negotiation.cc:407] Connection header received
0808 07:02:10.670872 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:10.670873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:10.670884 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:10.670939 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:02:10.671068 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.671637 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.672398 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.672643 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.672710 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:10.673168 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:02:10.673210 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:02:10.673272 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:10.673397 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 07:02:10.673425 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:02:10.673520 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:10.670562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:40383 (local address 127.13.207.1:52975)
0808 07:02:10.670631 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:10.670646 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:10.670798 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:10.670944 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:10.670944 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:10.670945 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:02:10.670997 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:10.670999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.671733 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.671734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:10.672312 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:10.672314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:10.672750 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:10.672752 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:10.672995 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:10.673012 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:02:10.673274 (+ 262us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:02:10.673278 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:02:10.673391 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 07:02:10.673449 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 07:02:10.686288 21926 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:02:10.686417 21926 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:02:10.687685 21926 catalog_manager.cc:5782] T 00000000000000000000000000000000 P befc8963088a49588ab2028974afc664: Sending DeleteTablet for 3 replicas of tablet 4d3acae5d8df4eac8ebb3814f1e9fd13
I20260808 07:02:10.688094 22217 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d3acae5d8df4eac8ebb3814f1e9fd13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:10 UTC) from {username='oryx', principal='oryx/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:33892
I20260808 07:02:10.688107 22069 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d3acae5d8df4eac8ebb3814f1e9fd13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:10 UTC) from {username='oryx', principal='oryx/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:52822
I20260808 07:02:10.688146 22364 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d3acae5d8df4eac8ebb3814f1e9fd13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:10 UTC) from {username='oryx', principal='oryx/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:60512
I20260808 07:02:10.688215 22490 tablet_replica.cc:330] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ef8811953fbe41e5821e087f2e58eab0: stopping tablet replica
I20260808 07:02:10.688242 22489 tablet_replica.cc:330] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ec60aadcbf224a4aabffcd82f789dc66: stopping tablet replica
I20260808 07:02:10.688282 22490 raft_consensus.cc:2229] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ef8811953fbe41e5821e087f2e58eab0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:02:10.688297 22489 raft_consensus.cc:2229] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ec60aadcbf224a4aabffcd82f789dc66 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:02:10.688328 22490 raft_consensus.cc:2258] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ef8811953fbe41e5821e087f2e58eab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:10.688352 22489 raft_consensus.cc:2258] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ec60aadcbf224a4aabffcd82f789dc66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:10.688477 22491 tablet_replica.cc:330] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea: stopping tablet replica
I20260808 07:02:10.688565 22491 raft_consensus.cc:2229] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:02:10.688668 22491 raft_consensus.cc:2258] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:10.688798 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 22003
I20260808 07:02:10.688867 22490 ts_tablet_manager.cc:1905] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ef8811953fbe41e5821e087f2e58eab0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:10.691741 22490 ts_tablet_manager.cc:1918] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ef8811953fbe41e5821e087f2e58eab0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:02:10.691854 22490 log.cc:1196] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ef8811953fbe41e5821e087f2e58eab0: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-1/wal/wals/4d3acae5d8df4eac8ebb3814f1e9fd13
I20260808 07:02:10.692142 22490 ts_tablet_manager.cc:1939] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P ef8811953fbe41e5821e087f2e58eab0: Deleting consensus metadata
I20260808 07:02:10.692260 22491 ts_tablet_manager.cc:1905] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:10.692566 21911 catalog_manager.cc:4841] TS ef8811953fbe41e5821e087f2e58eab0 (127.13.207.2:46755): tablet 4d3acae5d8df4eac8ebb3814f1e9fd13 (table test-table [id=b9adb1529e424391abea5b28d07f3bfb]) successfully deleted
I20260808 07:02:10.694051 22491 ts_tablet_manager.cc:1918] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:02:10.694141 22491 log.cc:1196] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/4d3acae5d8df4eac8ebb3814f1e9fd13
I20260808 07:02:10.694376 22491 ts_tablet_manager.cc:1939] T 4d3acae5d8df4eac8ebb3814f1e9fd13 P 41e555b6549a45168d2b12376f64daea: Deleting consensus metadata
I20260808 07:02:10.694929 21909 catalog_manager.cc:4841] TS 41e555b6549a45168d2b12376f64daea (127.13.207.3:40383): tablet 4d3acae5d8df4eac8ebb3814f1e9fd13 (table test-table [id=b9adb1529e424391abea5b28d07f3bfb]) successfully deleted
W20260808 07:02:10.698459 21911 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.207.1:38045 (error 108)
I20260808 07:02:10.698791 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 22148
I20260808 07:02:10.703691 22561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:10.698922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:38045 (local address 127.0.0.1:52844)
0808 07:02:10.703522 (+ 4600us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:10.703619 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.1:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4569,"thread_start_us":40,"threads_started":1}
W20260808 07:02:10.703914 21911 catalog_manager.cc:4584] TS ec60aadcbf224a4aabffcd82f789dc66 (127.13.207.1:38045): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4d3acae5d8df4eac8ebb3814f1e9fd13: Network error: Client connection negotiation failed: client connection to 127.13.207.1:38045: connect: Connection refused (error 111)
I20260808 07:02:10.709038 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 22297
I20260808 07:02:10.717110 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 21891
2026-08-08T07:02:10Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3307 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:02:10 dist-test-slave-8tdl krb5kdc[22564](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:02:10 dist-test-slave-8tdl krb5kdc[22564](info): set up 2 sockets
Aug 08 07:02:10 dist-test-slave-8tdl krb5kdc[22564](info): commencing operation
krb5kdc: starting...
W20260808 07:02:11.868201 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:02:11 dist-test-slave-8tdl krb5kdc[22564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:02:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:02:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:02:12.031085 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:41899
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:37029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:41899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:12.128290 22578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:12.128715 22578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:12.128772 22578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:12.130489 22578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:02:12.130605 22578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:12.130635 22578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:12.130666 22578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:02:12.130734 22578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:02:12.132614 22578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:37029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:41899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:41899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:12.132959 22578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:12.133247 22578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:12.139640 22584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:12.139640 22585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:12.139750 22578 server_base.cc:1034] running on GCE node
W20260808 07:02:12.139689 22587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:12.140717 22578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:12.141108 22578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:12.142422 22578 hybrid_clock.cc:648] HybridClock initialized: now 1786172532142368 us; error 69 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T07:02:12Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1430 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:02:12 dist-test-slave-8tdl krb5kdc[22592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:02:12 dist-test-slave-8tdl krb5kdc[22592](info): set up 2 sockets
Aug 08 07:02:12 dist-test-slave-8tdl krb5kdc[22592](info): commencing operation
krb5kdc: starting...
W20260808 07:02:13.299644 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:02:13 dist-test-slave-8tdl krb5kdc[22592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:02:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:02:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:02:13.452629 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:38637
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:38637
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:13.545370 22606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:13.545651 22606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:13.545682 22606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:13.547317 22606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:02:13.547389 22606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:13.547405 22606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:13.547421 22606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:02:13.547436 22606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:02:13.549407 22606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:38637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:38637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:13.549762 22606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:13.550182 22606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:13.556362 22612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:13.556416 22613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:13.556509 22606 server_base.cc:1034] running on GCE node
W20260808 07:02:13.556670 22615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:13.556986 22606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:13.557374 22606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:13.558543 22606 hybrid_clock.cc:648] HybridClock initialized: now 1786172533558497 us; error 60 us; skew 500 ppm
Aug 08 07:02:13 dist-test-slave-8tdl krb5kdc[22592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172533, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:13.561311 22606 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:02:13.561825 22606 webserver.cc:460] Webserver started at http://127.13.207.62:43501/ using document root <none> and password file <none>
I20260808 07:02:13.562080 22606 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:13.562151 22606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:13.562305 22606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:13.563496 22606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "5598414a061d4680960d9851334ba539"
format_stamp: "Formatted at 2026-08-08 07:02:13 on dist-test-slave-8tdl"
server_key: "ad37066f7bb8d808fde2c2e8ac46de24"
server_key_iv: "ae1e2990ad55540e80966bf06ac77a2d"
server_key_version: "encryptionkey@0"
I20260808 07:02:13.563881 22606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "5598414a061d4680960d9851334ba539"
format_stamp: "Formatted at 2026-08-08 07:02:13 on dist-test-slave-8tdl"
server_key: "ad37066f7bb8d808fde2c2e8ac46de24"
server_key_iv: "ae1e2990ad55540e80966bf06ac77a2d"
server_key_version: "encryptionkey@0"
I20260808 07:02:13.565379 22606 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:02:13.566375 22623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:13.566751 22606 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:02:13.566874 22606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "5598414a061d4680960d9851334ba539"
format_stamp: "Formatted at 2026-08-08 07:02:13 on dist-test-slave-8tdl"
server_key: "ad37066f7bb8d808fde2c2e8ac46de24"
server_key_iv: "ae1e2990ad55540e80966bf06ac77a2d"
server_key_version: "encryptionkey@0"
I20260808 07:02:13.567029 22606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:13.581907 22606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:13.584934 22606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:13.585232 22606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:13.590238 22606 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:38637
I20260808 07:02:13.590266 22675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:38637 every 8 connection(s)
I20260808 07:02:13.590731 22606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:02:13.591737 22676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:13.594509 22676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539: Bootstrap starting.
I20260808 07:02:13.595245 22676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:13.595526 22676 log.cc:824] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:13.596334 22676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539: No bootstrap required, opened a new log
I20260808 07:02:13.598028 22676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598414a061d4680960d9851334ba539" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38637 } }
I20260808 07:02:13.598189 22676 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:13.598214 22676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5598414a061d4680960d9851334ba539, State: Initialized, Role: FOLLOWER
I20260808 07:02:13.598330 22676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598414a061d4680960d9851334ba539" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38637 } }
I20260808 07:02:13.598421 22676 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:02:13.598461 22676 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:02:13.598546 22676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:13.599263 22676 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598414a061d4680960d9851334ba539" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38637 } }
I20260808 07:02:13.599404 22676 leader_election.cc:302] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5598414a061d4680960d9851334ba539; no voters:
I20260808 07:02:13.599728 22676 leader_election.cc:288] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:02:13.599799 22679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:02:13.600073 22676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:02:13.600240 22679 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [term 1 LEADER]: Becoming Leader. State: Replica: 5598414a061d4680960d9851334ba539, State: Running, Role: LEADER
I20260808 07:02:13.600323 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 22606
I20260808 07:02:13.600382 22679 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598414a061d4680960d9851334ba539" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38637 } }
I20260808 07:02:13.600433 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:02:13.600658 14140 external_mini_cluster.cc:1428] Setting key 871d2c455192f222d7c8e8c2866cf40e
I20260808 07:02:13.600935 22680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5598414a061d4680960d9851334ba539" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598414a061d4680960d9851334ba539" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38637 } } }
I20260808 07:02:13.601050 22680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:13.600942 22681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5598414a061d4680960d9851334ba539. Latest consensus state: current_term: 1 leader_uuid: "5598414a061d4680960d9851334ba539" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598414a061d4680960d9851334ba539" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38637 } } }
I20260808 07:02:13.601138 22681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [sys.catalog]: This master's current role is: LEADER
W20260808 07:02:13.602157 22696 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:02:13.602309 22696 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:02:13.602378 22689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:02:13.602563 22689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:02:13.606067 22689 catalog_manager.cc:1347] Generated new cluster ID: fed3ec71dfb7459dbc3557f29aed8154
I20260808 07:02:13.606142 22689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:02:13 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:02:13.613687 22693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:13.601351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52658 (local address 127.13.207.62:38637)
0808 07:02:13.601505 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:13.601516 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:13.601616 (+ 100us) server_negotiation.cc:407] Connection header received
0808 07:02:13.601790 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:13.601796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:13.601867 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:13.602044 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:13.602903 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:13.604191 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:13.605378 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:13.605993 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:13.606107 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:13.609497 (+ 3390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:13.609502 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:13.609503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:13.609517 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:13.611652 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:13.612157 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:13.612161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:13.612162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:13.612286 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:13.612516 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:13.612516 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:13.612517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:13.612847 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:13.612995 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:13.613137 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 07:02:13.613293 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":62,"threads_started":1}
I20260808 07:02:13.636545 22689 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:02:13.637347 22689 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:02:13.648624 22689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539: Generated new TSK 0
I20260808 07:02:13.648999 22689 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
I20260808 07:02:13.691107 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:38637
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:13.789143 22702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:13.789398 22702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:13.789427 22702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:13.790846 22702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:13.790906 22702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:13.790938 22702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:02:13.792622 22702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:38637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:13.792903 22702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:13.793164 22702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:02:13.799304 22702 server_base.cc:1034] running on GCE node
W20260808 07:02:13.799284 22708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:13.799284 22707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:13.799281 22712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:13.799870 22702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:13.800289 22702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:13.801486 22702 hybrid_clock.cc:648] HybridClock initialized: now 1786172533801443 us; error 94 us; skew 500 ppm
Aug 08 07:02:13 dist-test-slave-8tdl krb5kdc[22592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172533, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:13.805330 22702 init.cc:377] Logged in from keytab as kudu/127.13.207.1@KRBTEST.COM (short username kudu)
I20260808 07:02:13.805958 22702 webserver.cc:460] Webserver started at http://127.13.207.1:44833/ using document root <none> and password file <none>
I20260808 07:02:13.806233 22702 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:13.806315 22702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:13.806481 22702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:13.807737 22702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "0da2736d271e42a39d194f58e0b11805"
format_stamp: "Formatted at 2026-08-08 07:02:13 on dist-test-slave-8tdl"
server_key: "ab896ccae65b68cfd30baa3411c452b9"
server_key_iv: "57acf8fbca07ef817e71b55b15a6d707"
server_key_version: "encryptionkey@0"
I20260808 07:02:13.808123 22702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "0da2736d271e42a39d194f58e0b11805"
format_stamp: "Formatted at 2026-08-08 07:02:13 on dist-test-slave-8tdl"
server_key: "ab896ccae65b68cfd30baa3411c452b9"
server_key_iv: "57acf8fbca07ef817e71b55b15a6d707"
server_key_version: "encryptionkey@0"
I20260808 07:02:13.809651 22702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:02:13.810670 22719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:13.811214 22702 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:13.811371 22702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "0da2736d271e42a39d194f58e0b11805"
format_stamp: "Formatted at 2026-08-08 07:02:13 on dist-test-slave-8tdl"
server_key: "ab896ccae65b68cfd30baa3411c452b9"
server_key_iv: "57acf8fbca07ef817e71b55b15a6d707"
server_key_version: "encryptionkey@0"
I20260808 07:02:13.811492 22702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:13.840653 22702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:13.843173 22702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:13.843422 22702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:13.843703 22702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:13.844054 22702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:13.844113 22702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:13.844157 22702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:13.844187 22702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:13.852340 22702 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:40905
I20260808 07:02:13.852392 22837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:40905 every 8 connection(s)
I20260808 07:02:13.852787 22702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:02:13.858529 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 22702
I20260808 07:02:13.858636 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:02:13.858965 14140 external_mini_cluster.cc:1428] Setting key 81a346e0cc7142e5f921801e3bee7893
Aug 08 07:02:13 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172533, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:02:13.864219 22693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:13.853601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:57015 (local address 127.13.207.62:38637)
0808 07:02:13.853655 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:13.853657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:13.853886 (+ 229us) server_negotiation.cc:407] Connection header received
0808 07:02:13.854427 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:13.854428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:13.854439 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:13.854511 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:13.855782 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:13.856323 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:13.857567 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:13.857825 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:13.857908 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:13.860994 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:13.860996 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:13.860996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:13.861009 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:13.862584 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:13.863099 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:13.863101 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:13.863101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:13.863169 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:13.863455 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:13.863456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:13.863457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:13.863593 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:13.863672 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:13.864000 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 07:02:13.864116 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:02:13.864496 22840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:13.853539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.1:57015)
0808 07:02:13.853737 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:13.853789 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:13.854322 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:13.854608 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:13.854612 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:13.855379 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:13.855626 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:13.855632 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:13.856486 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:13.856487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:13.857458 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:13.857460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:13.858003 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:13.858004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:13.858144 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:13.859055 (+ 911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:13.859069 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:13.860873 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:13.862729 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:13.862741 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:13.862743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:13.863010 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:13.863243 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:13.863244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:13.863244 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:13.863377 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:13.863673 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:13.863682 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:13.863841 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 07:02:13.863949 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 07:02:13.864164 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":109,"threads_started":1}
I20260808 07:02:13.865118 22838 heartbeater.cc:347] Connected to a master server at 127.13.207.62:38637
I20260808 07:02:13.865216 22838 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:13.865440 22838 heartbeater.cc:511] Master 127.13.207.62:38637 requested a full tablet report, sending...
I20260808 07:02:13.866225 22640 ts_manager.cc:194] Registered new tserver with Master: 0da2736d271e42a39d194f58e0b11805 (127.13.207.1:40905)
I20260808 07:02:13.867296 22640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:57015
WARNING: no policy specified for kudu/127.13.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
I20260808 07:02:13.891899 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:38637
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:13.985968 22843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:13.986229 22843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:13.986259 22843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:13.987632 22843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:13.987694 22843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:13.987735 22843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:02:13.989418 22843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:38637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:13.989658 22843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:13.989854 22843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:13.995366 22849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:13.995383 22852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:13.995383 22850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:13.995678 22843 server_base.cc:1034] running on GCE node
I20260808 07:02:13.995976 22843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:13.996260 22843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:13.997493 22843 hybrid_clock.cc:648] HybridClock initialized: now 1786172533997489 us; error 43 us; skew 500 ppm
Aug 08 07:02:13 dist-test-slave-8tdl krb5kdc[22592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172533, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:14.001130 22843 init.cc:377] Logged in from keytab as kudu/127.13.207.2@KRBTEST.COM (short username kudu)
I20260808 07:02:14.002142 22843 webserver.cc:460] Webserver started at http://127.13.207.2:44971/ using document root <none> and password file <none>
I20260808 07:02:14.002480 22843 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:14.002576 22843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:14.002799 22843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:14.004081 22843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "df222505416a44bc9d1e67f84d9e8769"
format_stamp: "Formatted at 2026-08-08 07:02:13 on dist-test-slave-8tdl"
server_key: "c66d6d556e7d0bea802b80c54fc6ec91"
server_key_iv: "83b8a7b97523b213e5e482f63ad9b035"
server_key_version: "encryptionkey@0"
I20260808 07:02:14.004536 22843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "df222505416a44bc9d1e67f84d9e8769"
format_stamp: "Formatted at 2026-08-08 07:02:13 on dist-test-slave-8tdl"
server_key: "c66d6d556e7d0bea802b80c54fc6ec91"
server_key_iv: "83b8a7b97523b213e5e482f63ad9b035"
server_key_version: "encryptionkey@0"
I20260808 07:02:14.006507 22843 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260808 07:02:14.007826 22859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:14.008320 22843 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:14.008553 22843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "df222505416a44bc9d1e67f84d9e8769"
format_stamp: "Formatted at 2026-08-08 07:02:13 on dist-test-slave-8tdl"
server_key: "c66d6d556e7d0bea802b80c54fc6ec91"
server_key_iv: "83b8a7b97523b213e5e482f63ad9b035"
server_key_version: "encryptionkey@0"
I20260808 07:02:14.008661 22843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:14.019629 22843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:14.022666 22843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:14.022953 22843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:14.023259 22843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:14.023636 22843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:14.023698 22843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:14.023751 22843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:14.023779 22843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:14.031410 22843 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:42983
I20260808 07:02:14.031647 22979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:42983 every 8 connection(s)
I20260808 07:02:14.031903 22843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172533, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:02:14.039373 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 22843
I20260808 07:02:14.039673 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:02:14.039928 14140 external_mini_cluster.cc:1428] Setting key ec47477f445721c0aa01aaef65ecc6bb
I20260808 07:02:14.045502 22693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:14.032845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:44681 (local address 127.13.207.62:38637)
0808 07:02:14.032989 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:14.032991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:14.033093 (+ 102us) server_negotiation.cc:407] Connection header received
0808 07:02:14.033652 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:14.033653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:14.033666 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:14.033734 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:14.034999 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.035554 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.036756 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.037031 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.037203 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:14.040613 (+ 3410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:14.040614 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:14.040615 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:14.040625 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:14.043038 (+ 2413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:14.043701 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:14.043703 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:14.043703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:14.043780 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:14.044462 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:14.044463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:14.044464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:14.044623 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:14.044711 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:14.045089 (+ 378us) server_negotiation.cc:299] Negotiation successful
0808 07:02:14.045213 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260808 07:02:14.045923 22982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:14.032763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.2:44681)
0808 07:02:14.032950 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:14.033008 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:14.033544 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:14.033796 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:14.033800 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:14.034476 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:14.034795 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:14.034806 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.035675 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.035676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:14.036635 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:14.036638 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.037256 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.037257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:14.037385 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:14.038048 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:14.038057 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:14.040139 (+ 2082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:14.043202 (+ 3063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:14.043217 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:14.043219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:14.043565 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:14.043869 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:14.043870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:14.043870 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:14.044028 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:14.044714 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:14.044724 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:14.044899 (+ 175us) client_negotiation.cc:769] Sending connection context
0808 07:02:14.045047 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 07:02:14.045253 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":110,"threads_started":1}
I20260808 07:02:14.047006 22980 heartbeater.cc:347] Connected to a master server at 127.13.207.62:38637
I20260808 07:02:14.047194 22980 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:14.047477 22980 heartbeater.cc:511] Master 127.13.207.62:38637 requested a full tablet report, sending...
I20260808 07:02:14.048264 22640 ts_manager.cc:194] Registered new tserver with Master: df222505416a44bc9d1e67f84d9e8769 (127.13.207.2:42983)
I20260808 07:02:14.048708 22640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:44681
WARNING: no policy specified for kudu/127.13.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
I20260808 07:02:14.079732 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:38637
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:14.180823 22985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:14.181131 22985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:14.181176 22985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:14.182548 22985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:14.182638 22985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:14.182688 22985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:02:14.184450 22985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:38637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:14.184765 22985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:14.185038 22985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:14.190622 22990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:14.190586 22994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:14.190841 22991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:14.190881 22985 server_base.cc:1034] running on GCE node
I20260808 07:02:14.191205 22985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:14.191505 22985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:14.192714 22985 hybrid_clock.cc:648] HybridClock initialized: now 1786172534192692 us; error 40 us; skew 500 ppm
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172534, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:14.195988 22985 init.cc:377] Logged in from keytab as kudu/127.13.207.3@KRBTEST.COM (short username kudu)
I20260808 07:02:14.196542 22985 webserver.cc:460] Webserver started at http://127.13.207.3:42893/ using document root <none> and password file <none>
I20260808 07:02:14.196810 22985 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:14.196897 22985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:14.197094 22985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:14.198407 22985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "4e3bfde9edb4466b907d85eb84aea09b"
format_stamp: "Formatted at 2026-08-08 07:02:14 on dist-test-slave-8tdl"
server_key: "3dbbf196ea3eb5fdafc214723462210c"
server_key_iv: "3b301ffdab74fa1f0f6f1603dff32cfa"
server_key_version: "encryptionkey@0"
I20260808 07:02:14.198874 22985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "4e3bfde9edb4466b907d85eb84aea09b"
format_stamp: "Formatted at 2026-08-08 07:02:14 on dist-test-slave-8tdl"
server_key: "3dbbf196ea3eb5fdafc214723462210c"
server_key_iv: "3b301ffdab74fa1f0f6f1603dff32cfa"
server_key_version: "encryptionkey@0"
I20260808 07:02:14.200753 22985 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260808 07:02:14.201743 23001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:14.202117 22985 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:14.202239 22985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "4e3bfde9edb4466b907d85eb84aea09b"
format_stamp: "Formatted at 2026-08-08 07:02:14 on dist-test-slave-8tdl"
server_key: "3dbbf196ea3eb5fdafc214723462210c"
server_key_iv: "3b301ffdab74fa1f0f6f1603dff32cfa"
server_key_version: "encryptionkey@0"
I20260808 07:02:14.202335 22985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:14.222126 22985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:14.224787 22985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:14.224923 22985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:14.225186 22985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:14.225561 22985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:14.225603 22985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:14.225628 22985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:14.225641 22985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:14.232095 22985 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:44655
I20260808 07:02:14.232200 23119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:44655 every 8 connection(s)
I20260808 07:02:14.232575 22985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:02:14.236897 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 22985
I20260808 07:02:14.237030 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:02:14.237217 14140 external_mini_cluster.cc:1428] Setting key 1791dbbcc0149fd785e83e581e480b26
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172534, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:02:14.245596 22693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:14.233591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:39329 (local address 127.13.207.62:38637)
0808 07:02:14.233647 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:14.233649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:14.233905 (+ 256us) server_negotiation.cc:407] Connection header received
0808 07:02:14.234384 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:14.234386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:14.234397 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:14.234512 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:14.235739 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.236448 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.238587 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.238956 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.239040 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:14.242078 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:14.242080 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:14.242080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:14.242089 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:14.243765 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:14.244332 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:14.244334 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:14.244334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:14.244409 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:14.244802 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:14.244803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:14.244803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:14.244955 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:14.245022 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:14.245439 (+ 417us) server_negotiation.cc:299] Negotiation successful
0808 07:02:14.245521 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:02:14.245800 23122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:14.233494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.3:39329)
0808 07:02:14.233695 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:14.233742 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:14.234268 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:14.234590 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:14.234594 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:14.235373 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:14.235591 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:14.235597 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.236740 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.236741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:14.238333 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:14.238342 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.239072 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.239073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:14.239242 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:14.239940 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:14.239949 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:14.241852 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:14.243875 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:14.243884 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:14.243886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:14.244223 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:14.244552 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:14.244553 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:14.244553 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:14.244687 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:14.245070 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:14.245077 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:14.245237 (+ 160us) client_negotiation.cc:769] Sending connection context
0808 07:02:14.245351 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 07:02:14.245520 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":127,"threads_started":1}
I20260808 07:02:14.246409 23120 heartbeater.cc:347] Connected to a master server at 127.13.207.62:38637
I20260808 07:02:14.246537 23120 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:14.246840 23120 heartbeater.cc:511] Master 127.13.207.62:38637 requested a full tablet report, sending...
I20260808 07:02:14.247427 22640 ts_manager.cc:194] Registered new tserver with Master: 4e3bfde9edb4466b907d85eb84aea09b (127.13.207.3:44655)
I20260808 07:02:14.247838 22640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:39329
I20260808 07:02:14.250047 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.1.keytab.
Entry for principal oryx/127.13.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.1.keytab.
WARNING: no policy specified for oryx/127.13.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.2.keytab.
Entry for principal oryx/127.13.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.2.keytab.
WARNING: no policy specified for oryx/127.13.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.3.keytab.
Entry for principal oryx/127.13.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.3.keytab.
I20260808 07:02:14.343878 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 22702
I20260808 07:02:14.350386 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 22843
I20260808 07:02:14.357604 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 22985
I20260808 07:02:14.363529 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 22606
I20260808 07:02:14.371918 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:38637
--webserver_interface=127.13.207.62
--webserver_port=43501
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:38637
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:14.464252 23130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:14.464517 23130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:14.464548 23130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:14.465889 23130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:02:14.465950 23130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:14.465967 23130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:14.465983 23130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:02:14.465997 23130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:02:14.467846 23130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:38637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:38637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=43501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:14.468107 23130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:14.468312 23130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:14.473919 23135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:14.474084 23130 server_base.cc:1034] running on GCE node
W20260808 07:02:14.474000 23138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:14.473946 23136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:14.474555 23130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:14.474970 23130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:14.476181 23130 hybrid_clock.cc:648] HybridClock initialized: now 1786172534476152 us; error 42 us; skew 500 ppm
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172534, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:14.479218 23130 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:02:14.479743 23130 webserver.cc:460] Webserver started at http://127.13.207.62:43501/ using document root <none> and password file <none>
I20260808 07:02:14.479959 23130 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:14.480016 23130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:14.481540 23130 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:14.482322 23145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:14.482597 23130 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:14.482767 23130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "5598414a061d4680960d9851334ba539"
format_stamp: "Formatted at 2026-08-08 07:02:13 on dist-test-slave-8tdl"
server_key: "ad37066f7bb8d808fde2c2e8ac46de24"
server_key_iv: "ae1e2990ad55540e80966bf06ac77a2d"
server_key_version: "encryptionkey@0"
I20260808 07:02:14.483160 23130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:14.495174 23130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:14.497650 23130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:14.497782 23130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:14.502949 23130 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:38637
I20260808 07:02:14.503047 23201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:38637 every 8 connection(s)
I20260808 07:02:14.503448 23130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:02:14.505190 23202 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:02:14.505837 23202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539: Bootstrap starting.
I20260808 07:02:14.509486 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 23130
I20260808 07:02:14.509876 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:40905
--local_ip_for_outbound_sockets=127.13.207.1
--tserver_master_addrs=127.13.207.62:38637
--webserver_port=44833
--webserver_interface=127.13.207.1
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.207.1
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:02:14.515056 23202 log.cc:824] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:14.516934 23202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:02:14.517272 23202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539: Bootstrap complete.
I20260808 07:02:14.519829 23202 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598414a061d4680960d9851334ba539" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38637 } }
I20260808 07:02:14.520120 23202 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5598414a061d4680960d9851334ba539, State: Initialized, Role: FOLLOWER
I20260808 07:02:14.520354 23202 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598414a061d4680960d9851334ba539" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38637 } }
I20260808 07:02:14.520462 23202 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:02:14.520519 23202 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:02:14.520570 23202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:02:14.521706 23202 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598414a061d4680960d9851334ba539" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38637 } }
I20260808 07:02:14.521845 23202 leader_election.cc:302] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5598414a061d4680960d9851334ba539; no voters:
I20260808 07:02:14.522068 23202 leader_election.cc:288] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 07:02:14.522184 23206 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 07:02:14.522382 23206 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [term 2 LEADER]: Becoming Leader. State: Replica: 5598414a061d4680960d9851334ba539, State: Running, Role: LEADER
I20260808 07:02:14.522506 23202 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:02:14.522588 23206 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598414a061d4680960d9851334ba539" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38637 } }
I20260808 07:02:14.522989 23207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5598414a061d4680960d9851334ba539" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598414a061d4680960d9851334ba539" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38637 } } }
I20260808 07:02:14.523077 23207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:14.523490 23208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5598414a061d4680960d9851334ba539. Latest consensus state: current_term: 2 leader_uuid: "5598414a061d4680960d9851334ba539" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598414a061d4680960d9851334ba539" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38637 } } }
I20260808 07:02:14.523583 23208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:14.523741 23213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:02:14.524363 23213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:02:14.524587 23213 catalog_manager.cc:1259] Loaded cluster ID: fed3ec71dfb7459dbc3557f29aed8154
I20260808 07:02:14.524832 23213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:02:14.526109 23213 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:02:14.526290 23213 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5598414a061d4680960d9851334ba539: Loaded TSK: 0
I20260808 07:02:14.526516 23213 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:02:14.607359 23204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:14.607601 23204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:14.607630 23204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:14.609208 23204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:14.609292 23204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:14.609326 23204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:02:14.611261 23204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.1.keytab
--principal=oryx/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:40905
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=44833
--tserver_master_addrs=127.13.207.62:38637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:14.611680 23204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:14.611985 23204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:14.618952 23229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:14.619107 23232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:14.619170 23230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:14.619372 23204 server_base.cc:1034] running on GCE node
I20260808 07:02:14.619927 23204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:14.620253 23204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:14.621508 23204 hybrid_clock.cc:648] HybridClock initialized: now 1786172534621492 us; error 35 us; skew 500 ppm
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172534, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:14.625765 23204 init.cc:377] Logged in from keytab as oryx/127.13.207.1@KRBTEST.COM (short username oryx)
I20260808 07:02:14.626456 23204 webserver.cc:460] Webserver started at http://127.13.207.1:44833/ using document root <none> and password file <none>
I20260808 07:02:14.626830 23204 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:14.626943 23204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:14.629643 23204 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:02:14.630892 23239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:14.631218 23204 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:02:14.631345 23204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "0da2736d271e42a39d194f58e0b11805"
format_stamp: "Formatted at 2026-08-08 07:02:13 on dist-test-slave-8tdl"
server_key: "ab896ccae65b68cfd30baa3411c452b9"
server_key_iv: "57acf8fbca07ef817e71b55b15a6d707"
server_key_version: "encryptionkey@0"
I20260808 07:02:14.631778 23204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:14.652872 23204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:14.655841 23204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:14.656154 23204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:14.656466 23204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:14.656888 23204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:14.656950 23204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:14.656977 23204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:14.657027 23204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:14.664202 23204 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:40905
I20260808 07:02:14.664258 23360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:40905 every 8 connection(s)
I20260808 07:02:14.664691 23204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:02:14.667318 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 23204
I20260808 07:02:14.667902 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:42983
--local_ip_for_outbound_sockets=127.13.207.2
--tserver_master_addrs=127.13.207.62:38637
--webserver_port=44971
--webserver_interface=127.13.207.2
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.207.2
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:14.677136 23363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:14.665575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.1:54309)
0808 07:02:14.665826 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:14.665878 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:14.666815 (+ 937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:14.668030 (+ 1215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:14.668035 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:14.669103 (+ 1068us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:14.669463 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:14.669473 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.671454 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.671456 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:14.672422 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:14.672427 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.673095 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.673096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:14.673239 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:14.673922 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:14.673932 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:14.676818 (+ 2886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":114,"threads_started":1}
W20260808 07:02:14.677222 23364 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:14.665713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:54309 (local address 127.13.207.62:38637)
0808 07:02:14.665938 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:14.665947 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:14.665979 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:02:14.667275 (+ 1296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:14.667285 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:14.667360 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:14.667795 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:14.669717 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.671237 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.672545 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.672977 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.673115 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:14.676996 (+ 3881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.1:54309: BlockingRecv error: recv got EOF from 127.13.207.1:54309 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":83,"threads_started":1}
W20260808 07:02:14.678108 23361 heartbeater.cc:644] Failed to heartbeat to 127.13.207.62:38637 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.207.62:38637: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:14.684661 23363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:14.678431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.1:54213)
0808 07:02:14.678505 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:14.678520 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:14.678584 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:14.678810 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:14.678811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:14.679023 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:14.679106 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:14.679109 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.680082 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.680083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:14.681045 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:14.681049 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.681512 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.681513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:14.681601 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:14.682286 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:14.682292 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:14.684576 (+ 2284us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 07:02:14.684823 23364 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:14.678505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:54213 (local address 127.13.207.62:38637)
0808 07:02:14.678565 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:14.678572 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:14.678583 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:02:14.678659 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:14.678659 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:14.678718 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:14.678779 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:14.679436 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.679969 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.681173 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.681402 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.681481 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:14.684749 (+ 3268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.1:54213: BlockingRecv error: recv got EOF from 127.13.207.1:54213 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:14.690448 23363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:14.685110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.1:34903)
0808 07:02:14.685320 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:14.685330 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:14.685396 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:14.685618 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:14.685619 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:14.685768 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:14.685804 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:14.685806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.686881 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.686882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:14.687480 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:14.687482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.687940 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.687941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:14.687984 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:14.688513 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:14.688519 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:14.690388 (+ 1869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197}
W20260808 07:02:14.690632 23361 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:02:14.690802 23364 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:14.685163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:34903 (local address 127.13.207.62:38637)
0808 07:02:14.685229 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:14.685231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:14.685379 (+ 148us) server_negotiation.cc:407] Connection header received
0808 07:02:14.685462 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:14.685463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:14.685482 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:14.685591 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:14.685904 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.686783 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.687555 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.687868 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.687938 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:14.690527 (+ 2589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.1:34903: BlockingRecv error: recv got EOF from 127.13.207.1:34903 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260808 07:02:14.761883 23365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:14.762135 23365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:14.762163 23365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:14.763576 23365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:14.763640 23365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:14.763672 23365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:02:14.765297 23365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.2.keytab
--principal=oryx/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:42983
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=44971
--tserver_master_addrs=127.13.207.62:38637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:14.765579 23365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:14.765861 23365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:14.771607 23373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:14.771850 23365 server_base.cc:1034] running on GCE node
W20260808 07:02:14.771772 23371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:14.771868 23370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:14.772296 23365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:14.772650 23365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:14.773903 23365 hybrid_clock.cc:648] HybridClock initialized: now 1786172534773840 us; error 78 us; skew 500 ppm
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172534, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:14.776939 23365 init.cc:377] Logged in from keytab as oryx/127.13.207.2@KRBTEST.COM (short username oryx)
I20260808 07:02:14.777580 23365 webserver.cc:460] Webserver started at http://127.13.207.2:44971/ using document root <none> and password file <none>
I20260808 07:02:14.777860 23365 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:14.777935 23365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:14.779699 23365 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:02:14.780308 23381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:14.780494 23365 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:02:14.780596 23365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "df222505416a44bc9d1e67f84d9e8769"
format_stamp: "Formatted at 2026-08-08 07:02:13 on dist-test-slave-8tdl"
server_key: "c66d6d556e7d0bea802b80c54fc6ec91"
server_key_iv: "83b8a7b97523b213e5e482f63ad9b035"
server_key_version: "encryptionkey@0"
I20260808 07:02:14.780975 23365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:14.792599 23365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:14.795032 23365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:14.795215 23365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:14.795485 23365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:14.795816 23365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:14.795871 23365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:14.795912 23365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:14.795958 23365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:14.802446 23365 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:42983
I20260808 07:02:14.802491 23500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:42983 every 8 connection(s)
I20260808 07:02:14.802893 23365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:02:14.806414 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 23365
I20260808 07:02:14.806784 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:44655
--local_ip_for_outbound_sockets=127.13.207.3
--tserver_master_addrs=127.13.207.62:38637
--webserver_port=42893
--webserver_interface=127.13.207.3
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.207.3
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:14.811142 23503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:14.803696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.2:33153)
0808 07:02:14.803854 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:14.803891 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:14.804406 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:14.804686 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:14.804690 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:14.805286 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:14.805500 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:14.805505 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.806243 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.806244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:14.807295 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:14.807298 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.807711 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.807712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:14.807846 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:14.808478 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:14.808489 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:14.810977 (+ 2488us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
W20260808 07:02:14.811173 23364 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:14.803700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:33153 (local address 127.13.207.62:38637)
0808 07:02:14.803775 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:14.803777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:14.803988 (+ 211us) server_negotiation.cc:407] Connection header received
0808 07:02:14.804525 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:14.804526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:14.804542 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:14.804608 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:14.805615 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.806133 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.807391 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.807608 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.807699 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:14.811122 (+ 3423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.2:33153: BlockingRecv error: recv got EOF from 127.13.207.2:33153 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 07:02:14.811462 23501 heartbeater.cc:644] Failed to heartbeat to 127.13.207.62:38637 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.207.62:38637: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:14.816823 23503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:14.811753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.2:39483)
0808 07:02:14.811835 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:14.811850 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:14.811923 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:14.812179 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:14.812180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:14.812336 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:14.812451 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:14.812454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.813125 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.813126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:14.813629 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:14.813631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.814010 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.814011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:14.814092 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:14.814583 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:14.814589 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:14.816722 (+ 2133us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 07:02:14.816877 23364 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:14.811931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:39483 (local address 127.13.207.62:38637)
0808 07:02:14.811986 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:14.811989 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:14.811997 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:02:14.812024 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:14.812025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:14.812067 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:14.812154 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:14.812542 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.813039 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.813718 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.813936 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.814015 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:14.816811 (+ 2796us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.2:39483: BlockingRecv error: recv got EOF from 127.13.207.2:39483 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:14.822433 23503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:14.817140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.2:38557)
0808 07:02:14.817188 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:14.817198 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:14.817241 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:14.817412 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:14.817413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:14.817541 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:14.817572 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:14.817573 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.818563 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.818564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:14.819128 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:14.819130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.819648 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.819650 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:14.819695 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:14.820257 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:14.820262 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:14.822290 (+ 2028us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 07:02:14.822634 23501 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:02:14.822499 23364 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:14.817186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:38557 (local address 127.13.207.62:38637)
0808 07:02:14.817262 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:14.817265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:14.817276 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:02:14.817308 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:14.817309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:14.817322 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:14.817390 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:14.817648 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.818458 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.819219 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.819554 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.819637 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:14.822429 (+ 2792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.2:38557: BlockingRecv error: recv got EOF from 127.13.207.2:38557 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 07:02:14.900120 23504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:14.900393 23504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:14.900420 23504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:14.901703 23504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:14.901748 23504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:14.901778 23504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:02:14.903826 23504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/krb5kdc/oryx_127.13.207.3.keytab
--principal=oryx/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:44655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=42893
--tserver_master_addrs=127.13.207.62:38637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:14.904109 23504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:14.904423 23504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:14.910826 23510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:14.910986 23504 server_base.cc:1034] running on GCE node
W20260808 07:02:14.910826 23509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:14.911113 23512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:14.911438 23504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:14.911755 23504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:14.912957 23504 hybrid_clock.cc:648] HybridClock initialized: now 1786172534912939 us; error 33 us; skew 500 ppm
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172534, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:14.916049 23504 init.cc:377] Logged in from keytab as oryx/127.13.207.3@KRBTEST.COM (short username oryx)
I20260808 07:02:14.916642 23504 webserver.cc:460] Webserver started at http://127.13.207.3:42893/ using document root <none> and password file <none>
I20260808 07:02:14.916899 23504 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:14.917003 23504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:14.918548 23504 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:02:14.919452 23520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:14.919871 23504 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:14.920073 23504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "4e3bfde9edb4466b907d85eb84aea09b"
format_stamp: "Formatted at 2026-08-08 07:02:14 on dist-test-slave-8tdl"
server_key: "3dbbf196ea3eb5fdafc214723462210c"
server_key_iv: "3b301ffdab74fa1f0f6f1603dff32cfa"
server_key_version: "encryptionkey@0"
I20260808 07:02:14.920480 23504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:14.931046 23504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:14.933707 23504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:14.933903 23504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:14.934192 23504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:14.934538 23504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:14.934595 23504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:14.934639 23504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:14.934669 23504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:14.943181 23504 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:44655
I20260808 07:02:14.943341 23638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:44655 every 8 connection(s)
I20260808 07:02:14.943727 23504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:02:14.945134 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 23504
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
W20260808 07:02:14.955080 23364 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:14.944827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:36809 (local address 127.13.207.62:38637)
0808 07:02:14.944932 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:14.944935 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:14.945100 (+ 165us) server_negotiation.cc:407] Connection header received
0808 07:02:14.945979 (+ 879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:14.945981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:14.945996 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:14.946078 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:14.947728 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.948530 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.949931 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.950355 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.950435 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:14.955007 (+ 4572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.3:36809: BlockingRecv error: recv got EOF from 127.13.207.3:36809 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:02:14.955338 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:14.944748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.3:36809)
0808 07:02:14.944940 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:14.944994 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:14.945730 (+ 736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:14.946245 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:14.946254 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:14.947201 (+ 947us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:14.947602 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:14.947610 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.948644 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.948645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:14.949832 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:14.949835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.950597 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.950598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:14.950794 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:14.951566 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:14.951577 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:14.954801 (+ 3224us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":187,"threads_started":1}
W20260808 07:02:14.955793 23639 heartbeater.cc:644] Failed to heartbeat to 127.13.207.62:38637 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.207.62:38637: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
I20260808 07:02:14.956609 23643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:14.946138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38522 (local address 127.13.207.62:38637)
0808 07:02:14.946276 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:14.946278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:14.946284 (+ 6us) server_negotiation.cc:407] Connection header received
0808 07:02:14.946300 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:14.946301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:14.946309 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:14.946365 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:14.947327 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.948233 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.949434 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.949670 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.949735 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:14.951701 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:14.951706 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:14.951707 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:14.951718 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:14.953983 (+ 2265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:14.954508 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:14.954513 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:14.954514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:14.954630 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:14.955135 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:14.955136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:14.955137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:14.955435 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:14.955608 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:14.956193 (+ 585us) server_negotiation.cc:299] Negotiation successful
0808 07:02:14.956357 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:14.962093 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:14.956398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.3:43147)
0808 07:02:14.956464 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:14.956478 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:14.956526 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:14.956859 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:14.956860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:14.957048 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:14.957110 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:14.957112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.958218 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.958219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:14.958903 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:14.958906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.959295 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.959296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:14.959371 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:14.959864 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:14.959869 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:14.962009 (+ 2140us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 07:02:14.962268 23364 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:14.956475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:43147 (local address 127.13.207.62:38637)
0808 07:02:14.956618 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:14.956620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:14.956634 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:02:14.956657 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:14.956658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:14.956688 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:14.956777 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:14.957203 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.958126 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.959002 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.959201 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.959291 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:14.962191 (+ 2900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.3:43147: BlockingRecv error: recv got EOF from 127.13.207.3:43147 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:14 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:14.967870 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:14.962801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.3:50055)
0808 07:02:14.962862 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:14.962875 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:14.962932 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:14.963089 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:14.963090 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:14.963233 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:14.963274 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:14.963276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.964053 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.964054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:14.964620 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:14.964622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.965025 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.965026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:14.965074 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:14.965583 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:14.965589 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:14.967806 (+ 2217us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 07:02:14.968046 23639 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:02:14.968092 23364 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:14.962838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:50055 (local address 127.13.207.62:38637)
0808 07:02:14.962912 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:14.962914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:14.962923 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:02:14.963002 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:14.963003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:14.963012 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:14.963067 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:14.963374 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.963919 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.964721 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:14.964909 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:14.964975 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:14.968030 (+ 3055us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.3:50055: BlockingRecv error: recv got EOF from 127.13.207.3:50055 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 07:02:15 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:15 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:15.696823 23645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:15.691089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.1:38407)
0808 07:02:15.691323 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:15.691336 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:15.691445 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:15.691635 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:15.691636 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:15.691851 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:15.691901 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:15.691903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:15.692720 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:15.692722 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:15.693349 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:15.693352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:15.693793 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:15.693794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:15.693863 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:15.694464 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:15.694469 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:15.696699 (+ 2230us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":173,"threads_started":1}
W20260808 07:02:15.696905 23644 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:15.691163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:38407 (local address 127.13.207.62:38637)
0808 07:02:15.691321 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:15.691324 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:15.691425 (+ 101us) server_negotiation.cc:407] Connection header received
0808 07:02:15.691522 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:15.691523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:15.691535 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:15.691628 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:15.691995 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:15.692600 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:15.693462 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:15.693700 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:15.693791 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:15.696825 (+ 3034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.1:38407: BlockingRecv error: recv got EOF from 127.13.207.1:38407 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":89,"threads_started":1}
Aug 08 07:02:15 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:15 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:15.829345 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:15.823120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.2:39797)
0808 07:02:15.823362 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:15.823376 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:15.823468 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:15.823639 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:15.823640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:15.823847 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:15.823900 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:15.823902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:15.824840 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:15.824842 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:15.825623 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:15.825626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:15.826054 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:15.826054 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:15.826118 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:15.826847 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:15.826853 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:15.829214 (+ 2361us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":110,"threads_started":1}
W20260808 07:02:15.829495 23644 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:15.823221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:39797 (local address 127.13.207.62:38637)
0808 07:02:15.823346 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:15.823349 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:15.823455 (+ 106us) server_negotiation.cc:407] Connection header received
0808 07:02:15.823550 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:15.823551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:15.823566 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:15.823620 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:15.823995 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:15.824734 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:15.825722 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:15.825964 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:15.826054 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:15.829321 (+ 3267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.2:39797: BlockingRecv error: recv got EOF from 127.13.207.2:39797 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 07:02:15 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:15 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:15.974684 23647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:15.968668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.3:37157)
0808 07:02:15.968869 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:15.968881 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:15.968951 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:15.969147 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:15.969148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:15.969326 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:15.969379 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:15.969380 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:15.970334 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:15.970335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:15.971184 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:15.971188 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:15.971663 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:15.971664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:15.971732 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:15.972434 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:15.972439 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:15.974562 (+ 2123us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":105,"threads_started":1}
W20260808 07:02:15.974774 23644 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:15.968754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:37157 (local address 127.13.207.62:38637)
0808 07:02:15.968873 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:15.968876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:15.968939 (+ 63us) server_negotiation.cc:407] Connection header received
0808 07:02:15.969009 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:15.969010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:15.969027 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:15.969105 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:15.969572 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:15.970189 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:15.971326 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:15.971567 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:15.971662 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:15.974666 (+ 3004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.3:37157: BlockingRecv error: recv got EOF from 127.13.207.3:37157 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 08 07:02:16 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:16 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:16.703547 23648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:16.697606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.1:43097)
0808 07:02:16.697837 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:16.697899 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:16.697981 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:16.698184 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:16.698185 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:16.698360 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:16.698411 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:16.698412 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:16.699313 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:16.699314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:16.699877 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:16.699881 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:16.700287 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:16.700294 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:16.700349 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:16.700961 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:16.700966 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:16.703437 (+ 2471us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":135,"threads_started":1}
W20260808 07:02:16.703672 23649 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:16.697762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:43097 (local address 127.13.207.62:38637)
0808 07:02:16.697919 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:16.697921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:16.697977 (+ 56us) server_negotiation.cc:407] Connection header received
0808 07:02:16.698043 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:16.698044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:16.698057 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:16.698109 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:16.698545 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:16.699169 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:16.699989 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:16.700188 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:16.700251 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:16.703562 (+ 3311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.1:43097: BlockingRecv error: recv got EOF from 127.13.207.1:43097 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":95,"threads_started":1}
Aug 08 07:02:16 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:16 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:16.835366 23650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:16.830071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.2:35553)
0808 07:02:16.830253 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:16.830268 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:16.830362 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:16.830513 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:16.830514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:16.830762 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:16.830822 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:16.830824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:16.831551 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:16.831552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:16.832089 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:16.832091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:16.832485 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:16.832486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:16.832547 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:16.833120 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:16.833126 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:16.835254 (+ 2128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
W20260808 07:02:16.835443 23649 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:16.830214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:35553 (local address 127.13.207.62:38637)
0808 07:02:16.830295 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:16.830298 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:16.830355 (+ 57us) server_negotiation.cc:407] Connection header received
0808 07:02:16.830435 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:16.830436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:16.830450 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:16.830496 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:16.830903 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:16.831453 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:16.832190 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:16.832406 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:16.832486 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:16.835366 (+ 2880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.2:35553: BlockingRecv error: recv got EOF from 127.13.207.2:35553 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 07:02:16 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:16 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:16.981215 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:16.975417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.3:38941)
0808 07:02:16.975619 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:16.975630 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:16.975680 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:16.975851 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:16.975852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:16.976022 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:16.976072 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:16.976073 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:16.976891 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:16.976892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:16.977519 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:16.977521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:16.977961 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:16.977962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:16.978015 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:16.978586 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:16.978591 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:16.981152 (+ 2561us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":102,"threads_started":1}
W20260808 07:02:16.981395 23649 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:16.975596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:38941 (local address 127.13.207.62:38637)
0808 07:02:16.975697 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:16.975699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:16.975712 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:02:16.975742 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:16.975742 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:16.975756 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:16.975833 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:16.976186 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:16.976772 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:16.977632 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:16.977853 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:16.977924 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:16.981290 (+ 3366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.3:38941: BlockingRecv error: recv got EOF from 127.13.207.3:38941 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 08 07:02:17 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:17 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:17.715169 23652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:17.704185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.1:60437)
0808 07:02:17.704387 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:17.704403 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:17.704526 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:17.704798 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:17.704799 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:17.704988 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:17.705053 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:17.705055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:17.705771 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:17.705773 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:17.706374 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:17.706376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:17.706924 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:17.706926 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:17.706982 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:17.712691 (+ 5709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:17.712698 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:17.715078 (+ 2380us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":73,"threads_started":1}
W20260808 07:02:17.715267 23653 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:17.704480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:60437 (local address 127.13.207.62:38637)
0808 07:02:17.704638 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:17.704640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:17.704649 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:02:17.704675 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:17.704675 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:17.704685 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:17.704757 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:17.705162 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:17.705683 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:17.706492 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:17.706755 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:17.706891 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:17.715192 (+ 8301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.1:60437: BlockingRecv error: recv got EOF from 127.13.207.1:60437 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":96,"threads_started":1}
Aug 08 07:02:17 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:17 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:17.842288 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:17.836354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.2:54673)
0808 07:02:17.836615 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:17.836635 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:17.836701 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:17.836932 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:17.836937 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:17.837165 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:17.837216 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:17.837218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:17.837982 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:17.837983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:17.838654 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:17.838657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:17.839098 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:17.839098 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:17.839147 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:17.839694 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:17.839699 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:17.842216 (+ 2517us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":155,"threads_started":1}
W20260808 07:02:17.842589 23653 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:17.836460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:54673 (local address 127.13.207.62:38637)
0808 07:02:17.836567 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:17.836570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:17.836785 (+ 215us) server_negotiation.cc:407] Connection header received
0808 07:02:17.836819 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:17.836820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:17.836833 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:17.836896 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:17.837310 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:17.837885 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:17.838808 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:17.839022 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:17.839085 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:17.842540 (+ 3455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.2:54673: BlockingRecv error: recv got EOF from 127.13.207.2:54673 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 07:02:17 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:17 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:17.987798 23655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:17.981859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.3:60247)
0808 07:02:17.982080 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:17.982093 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:17.982179 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:17.982393 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:17.982394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:17.982570 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:17.982622 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:17.982624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:17.983545 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:17.983546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:17.984124 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:17.984127 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:17.984617 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:17.984619 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:17.984694 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:17.985375 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:17.985381 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:17.987736 (+ 2355us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":122,"threads_started":1}
W20260808 07:02:17.987990 23653 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:17.982062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:60247 (local address 127.13.207.62:38637)
0808 07:02:17.982117 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:17.982119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:17.982175 (+ 56us) server_negotiation.cc:407] Connection header received
0808 07:02:17.982233 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:17.982234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:17.982248 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:17.982299 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:17.982785 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:17.983419 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:17.984244 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:17.984463 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:17.984619 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:17.987911 (+ 3292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.3:60247: BlockingRecv error: recv got EOF from 127.13.207.3:60247 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 08 07:02:18 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:18 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:18.721606 23656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:18.715738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.1:58521)
0808 07:02:18.715935 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:18.715951 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:18.716013 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:18.716251 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:18.716252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:18.716448 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:18.716513 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:18.716515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:18.717512 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:18.717513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:18.718117 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:18.718120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:18.718576 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:18.718577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:18.718631 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:18.719276 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:18.719283 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:18.721538 (+ 2255us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":111,"threads_started":1}
W20260808 07:02:18.721823 23657 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:18.715865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:58521 (local address 127.13.207.62:38637)
0808 07:02:18.716083 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:18.716084 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:18.716094 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:02:18.716122 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:18.716123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:18.716135 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:18.716198 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:18.716699 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:18.717256 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:18.718228 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:18.718441 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:18.718531 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:18.721729 (+ 3198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.1:58521: BlockingRecv error: recv got EOF from 127.13.207.1:58521 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":152,"threads_started":1}
Aug 08 07:02:18 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:18 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:18.849287 23658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:18.843038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.2:51237)
0808 07:02:18.843209 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:18.843230 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:18.843291 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:18.843538 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:18.843540 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:18.843742 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:18.843793 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:18.843794 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:18.844797 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:18.844798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:18.845507 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:18.845511 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:18.846005 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:18.846006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:18.846065 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:18.846896 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:18.846902 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:18.849221 (+ 2319us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":108,"threads_started":1}
W20260808 07:02:18.849766 23657 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:18.843158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:51237 (local address 127.13.207.62:38637)
0808 07:02:18.843218 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:18.843219 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:18.843364 (+ 145us) server_negotiation.cc:407] Connection header received
0808 07:02:18.843405 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:18.843406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:18.843420 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:18.843494 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:18.843896 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:18.844684 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:18.845628 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:18.845863 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:18.845946 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:18.849460 (+ 3514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.2:51237: BlockingRecv error: recv got EOF from 127.13.207.2:51237 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 08 07:02:18 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:18 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:18.993871 23659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:18.988449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.3:55459)
0808 07:02:18.988653 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:18.988665 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:18.988720 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:18.988945 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:18.988946 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:18.989161 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:18.989214 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:18.989216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:18.989985 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:18.989987 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:18.990539 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:18.990540 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:18.991104 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:18.991105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:18.991160 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:18.991815 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:18.991820 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:18.993807 (+ 1987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":129,"threads_started":1}
W20260808 07:02:18.994364 23657 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:18.988621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:55459 (local address 127.13.207.62:38637)
0808 07:02:18.988703 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:18.988705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:18.988718 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:02:18.988775 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:18.988776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:18.988790 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:18.988863 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:18.989368 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:18.989889 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:18.990738 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:18.990988 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:18.991074 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:18.994284 (+ 3210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.3:55459: BlockingRecv error: recv got EOF from 127.13.207.3:55459 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 07:02:19 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:19 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:19.731033 23660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:19.722429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.1:52177)
0808 07:02:19.722611 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:19.722625 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:19.722760 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:19.722985 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:19.722986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:19.723340 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:19.723424 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:19.723428 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:19.724862 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:19.724864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:19.726151 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:19.726157 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:19.726867 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:19.726869 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:19.726969 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:19.727588 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:19.727595 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:19.730934 (+ 3339us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":100,"threads_started":1}
W20260808 07:02:19.731304 23661 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:19.722588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:52177 (local address 127.13.207.62:38637)
0808 07:02:19.722775 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:19.722777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:19.722791 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:02:19.722832 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:19.722833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:19.722851 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:19.722955 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:19.723582 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:19.724698 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:19.726335 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:19.726755 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:19.726831 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:19.731215 (+ 4384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.1:52177: BlockingRecv error: recv got EOF from 127.13.207.1:52177 (error 108)
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":74,"threads_started":1}
Aug 08 07:02:19 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:19 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:19.856048 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:19.850160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.2:48587)
0808 07:02:19.850395 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:19.850407 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:19.850482 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:19.850722 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:19.850724 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:19.850918 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:19.850973 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:19.850974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:19.851822 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:19.851823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:19.852371 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:19.852373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:19.852885 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:19.852887 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:19.852953 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:19.853599 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:19.853605 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:19.855978 (+ 2373us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":110,"threads_started":1}
W20260808 07:02:19.856256 23661 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:19.850442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:48587 (local address 127.13.207.62:38637)
0808 07:02:19.850528 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:19.850530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:19.850540 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:02:19.850576 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:19.850577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:19.850589 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:19.850665 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:19.851077 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:19.851673 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:19.852488 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:19.852702 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:19.852851 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:19.856171 (+ 3320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.2:48587: BlockingRecv error: recv got EOF from 127.13.207.2:48587 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 07:02:19 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:19 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:19.999967 23663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:19.994529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.3:52819)
0808 07:02:19.994719 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:19.994731 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:19.994803 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:19.995061 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:19.995061 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:19.995282 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:19.995344 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:19.995346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:19.996168 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:19.996169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:19.996712 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:19.996714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:19.997225 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:19.997226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:19.997275 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:19.997831 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:19.997836 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:19.999902 (+ 2066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":72,"threads_started":1}
W20260808 07:02:20.000290 23661 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:19.994600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:52819 (local address 127.13.207.62:38637)
0808 07:02:19.994714 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:19.994717 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:19.994805 (+ 88us) server_negotiation.cc:407] Connection header received
0808 07:02:19.994952 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:19.994954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:19.994974 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:19.995054 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:19.995484 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:19.996058 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:19.996840 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:19.997096 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:19.997167 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:20.000223 (+ 3056us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.3:52819: BlockingRecv error: recv got EOF from 127.13.207.3:52819 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 07:02:20 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:20 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:20.737721 23664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:20.732425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.1:36627)
0808 07:02:20.732559 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:20.732572 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:20.732636 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:20.732861 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:20.732862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:20.733033 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:20.733082 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:20.733084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:20.733781 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:20.733781 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:20.734355 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:20.734357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:20.734838 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:20.734839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:20.734893 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:20.735405 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:20.735410 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:20.737656 (+ 2246us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":76,"threads_started":1}
W20260808 07:02:20.738003 23665 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:20.732533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:36627 (local address 127.13.207.62:38637)
0808 07:02:20.732718 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:20.732719 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:20.732729 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:02:20.732754 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:20.732754 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:20.732765 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:20.732838 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:20.733160 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:20.733697 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:20.734484 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:20.734750 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:20.734838 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:20.737932 (+ 3094us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.1:36627: BlockingRecv error: recv got EOF from 127.13.207.1:36627 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":114,"threads_started":1}
Aug 08 07:02:20 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:20 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:20.863308 23666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:20.856683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.2:50803)
0808 07:02:20.856956 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:20.856968 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:20.857071 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:20.857265 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:20.857266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:20.857502 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:20.857550 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:20.857552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:20.858508 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:20.858509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:20.859262 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:20.859265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:20.859715 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:20.859716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:20.859781 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:20.860561 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:20.860566 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:20.863236 (+ 2670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":124,"threads_started":1}
W20260808 07:02:20.863659 23665 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:20.856800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:50803 (local address 127.13.207.62:38637)
0808 07:02:20.856942 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:20.856946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:20.857050 (+ 104us) server_negotiation.cc:407] Connection header received
0808 07:02:20.857157 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:20.857158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:20.857174 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:20.857241 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:20.857641 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:20.858219 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:20.859361 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:20.859586 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:20.859712 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:20.863583 (+ 3871us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.2:50803: BlockingRecv error: recv got EOF from 127.13.207.2:50803 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Aug 08 07:02:20 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:21 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:21.007555 23667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:21.000794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.3:53123)
0808 07:02:21.000996 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:21.001009 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:21.001072 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:21.001343 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:21.001345 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:21.001630 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:21.001712 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:21.001715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:21.002829 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:21.002831 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:21.003673 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:21.003678 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:21.004189 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:21.004190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:21.004249 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:21.005070 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:21.005075 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:21.007476 (+ 2401us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":123,"threads_started":1}
W20260808 07:02:21.007771 23665 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:21.000941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:53123 (local address 127.13.207.62:38637)
0808 07:02:21.001045 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:21.001047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:21.001068 (+ 21us) server_negotiation.cc:407] Connection header received
0808 07:02:21.001146 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:21.001146 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:21.001166 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:21.001263 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:21.001860 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:21.002648 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:21.003832 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:21.004062 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:21.004139 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:21.007688 (+ 3549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.3:53123: BlockingRecv error: recv got EOF from 127.13.207.3:53123 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 08 07:02:21 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:21 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:21.744534 23668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:21.738509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.1:43733)
0808 07:02:21.738753 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:21.738768 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:21.738850 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:21.739061 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:21.739062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:21.739337 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:21.739395 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:21.739398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:21.740170 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:21.740171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:21.740927 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:21.740930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:21.741308 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:21.741308 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:21.741350 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:21.741903 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:21.741908 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:21.744425 (+ 2517us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":109,"threads_started":1}
W20260808 07:02:21.744623 23669 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:21.738613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:43733 (local address 127.13.207.62:38637)
0808 07:02:21.738798 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:21.738800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:21.738833 (+ 33us) server_negotiation.cc:407] Connection header received
0808 07:02:21.738964 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:21.738964 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:21.738976 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:21.739036 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:21.739515 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:21.740068 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:21.741035 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:21.741239 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:21.741297 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:21.744539 (+ 3242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.1:43733: BlockingRecv error: recv got EOF from 127.13.207.1:43733 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":125,"threads_started":1}
Aug 08 07:02:21 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:21 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:21.869808 23670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:21.863979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.2:47359)
0808 07:02:21.864208 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:21.864222 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:21.864365 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:21.864544 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:21.864545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:21.864727 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:21.864779 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:21.864780 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:21.865653 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:21.865654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:21.866257 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:21.866259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:21.866770 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:21.866771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:21.866840 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:21.867404 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:21.867412 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:21.869739 (+ 2327us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":119,"threads_started":1}
W20260808 07:02:21.870083 23669 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:21.864150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:47359 (local address 127.13.207.62:38637)
0808 07:02:21.864273 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:21.864275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:21.864410 (+ 135us) server_negotiation.cc:407] Connection header received
0808 07:02:21.864444 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:21.864445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:21.864460 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:21.864518 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:21.864977 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:21.865543 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:21.866363 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:21.866584 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:21.866659 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:21.869940 (+ 3281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.2:47359: BlockingRecv error: recv got EOF from 127.13.207.2:47359 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 07:02:22 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:22 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:22.014775 23671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:22.008370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.3:39219)
0808 07:02:22.008579 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:22.008593 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:22.008697 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:22.009019 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:22.009020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:22.009382 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:22.009435 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:22.009438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:22.010476 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:22.010477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:22.011146 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:22.011149 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:22.011617 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:22.011618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:22.011669 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:22.012219 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:22.012224 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:22.014650 (+ 2426us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":109,"threads_started":1}
W20260808 07:02:22.015057 23669 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:22.008482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:39219 (local address 127.13.207.62:38637)
0808 07:02:22.008564 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:22.008566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:22.008754 (+ 188us) server_negotiation.cc:407] Connection header received
0808 07:02:22.008795 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:22.008796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:22.008875 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:22.008960 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:22.009724 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:22.010362 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:22.011300 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:22.011525 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:22.011593 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:22.014910 (+ 3317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.3:39219: BlockingRecv error: recv got EOF from 127.13.207.3:39219 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 07:02:22 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:22 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:22.757958 23672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:22.745221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.1:51585)
0808 07:02:22.745488 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:22.745507 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:22.745591 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:22.745754 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:22.745755 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:22.745953 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:22.746007 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:22.746009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:22.746930 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:22.746931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:22.747648 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:22.747652 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:22.748140 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:22.748141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:22.748195 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:22.755284 (+ 7089us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:22.755291 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:22.757870 (+ 2579us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":80,"threads_started":1}
W20260808 07:02:22.758155 23673 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:22.745407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:51585 (local address 127.13.207.62:38637)
0808 07:02:22.745582 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:22.745584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:22.745595 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:02:22.745643 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:22.745643 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:22.745655 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:22.745724 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:22.746120 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:22.746836 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:22.747789 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:22.748036 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:22.748126 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:22.758030 (+ 9904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.1:51585: BlockingRecv error: recv got EOF from 127.13.207.1:51585 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":81,"threads_started":1}
Aug 08 07:02:22 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:22 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:22.876096 23674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:22.870545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.2:36641)
0808 07:02:22.870788 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:22.870800 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:22.870875 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:22.871011 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:22.871012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:22.871185 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:22.871236 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:22.871237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:22.872239 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:22.872240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:22.872778 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:22.872781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:22.873236 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:22.873237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:22.873288 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:22.873798 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:22.873802 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:22.876025 (+ 2223us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":154,"threads_started":1}
W20260808 07:02:22.876286 23673 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:22.870603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:36641 (local address 127.13.207.62:38637)
0808 07:02:22.870704 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:22.870707 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:22.870877 (+ 170us) server_negotiation.cc:407] Connection header received
0808 07:02:22.870936 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:22.870936 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:22.870951 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:22.870997 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:22.871305 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:22.872143 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:22.872917 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:22.873139 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:22.873215 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:22.876209 (+ 2994us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.2:36641: BlockingRecv error: recv got EOF from 127.13.207.2:36641 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 07:02:23 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:23 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:23.021198 23675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:23.015501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.3:56505)
0808 07:02:23.015650 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:23.015666 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:23.015732 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:23.015984 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:23.015985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:23.016159 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:23.016208 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:23.016209 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:23.017090 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:23.017091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:23.017642 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:23.017643 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:23.018230 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:23.018237 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:23.018292 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:23.018858 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:23.018864 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:23.021118 (+ 2254us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
W20260808 07:02:23.021335 23673 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:23.015631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:56505 (local address 127.13.207.62:38637)
0808 07:02:23.015679 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:23.015681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:23.015728 (+ 47us) server_negotiation.cc:407] Connection header received
0808 07:02:23.015778 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:23.015779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:23.015791 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:23.015856 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:23.016322 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:23.016849 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:23.017740 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:23.017989 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:23.018053 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:23.021278 (+ 3225us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.3:56505: BlockingRecv error: recv got EOF from 127.13.207.3:56505 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 08 07:02:23 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:23 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:23.764770 23676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:23.758723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.1:40945)
0808 07:02:23.758882 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:23.758892 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:23.759000 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:23.759196 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:23.759197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:23.759381 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:23.759434 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:23.759436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:23.760303 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:23.760305 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:23.760962 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:23.760965 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:23.761483 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:23.761485 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:23.761543 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:23.762136 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:23.762142 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:23.764666 (+ 2524us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":77,"threads_started":1}
W20260808 07:02:23.764840 23677 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:23.758808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:40945 (local address 127.13.207.62:38637)
0808 07:02:23.758963 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:23.758965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:23.758978 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:02:23.759071 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:23.759071 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:23.759091 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:23.759177 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:23.759524 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:23.760151 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:23.761125 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:23.761365 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:23.761445 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:23.764767 (+ 3322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.1:40945: BlockingRecv error: recv got EOF from 127.13.207.1:40945 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
Aug 08 07:02:23 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:23 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:23.883337 23679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:23.876807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.2:35405)
0808 07:02:23.877055 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:23.877066 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:23.877291 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:23.877533 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:23.877534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:23.877802 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:23.877897 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:23.877899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:23.878823 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:23.878824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:23.879366 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:23.879368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:23.879807 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:23.879808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:23.879884 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:23.880506 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:23.880512 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:23.883181 (+ 2669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":149,"threads_started":1}
W20260808 07:02:23.883635 23677 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:23.876894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:35405 (local address 127.13.207.62:38637)
0808 07:02:23.876951 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:23.876960 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:23.877332 (+ 372us) server_negotiation.cc:407] Connection header received
0808 07:02:23.877377 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:23.877378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:23.877396 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:23.877479 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:23.878013 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:23.878654 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:23.879492 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:23.879697 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:23.879762 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:23.883550 (+ 3788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.2:35405: BlockingRecv error: recv got EOF from 127.13.207.2:35405 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 07:02:24 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:24 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:24.028381 23680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:24.022061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.3:48453)
0808 07:02:24.022228 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:24.022248 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:24.022306 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:24.022523 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:24.022525 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:24.022820 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:24.022880 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:24.022882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:24.023798 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:24.023799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:24.024740 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:24.024743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:24.025174 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:24.025175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:24.025230 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:24.025768 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:24.025774 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:24.028238 (+ 2464us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":102,"threads_started":1}
W20260808 07:02:24.028689 23677 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:24.022262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:48453 (local address 127.13.207.62:38637)
0808 07:02:24.022328 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:24.022330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:24.022344 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:02:24.022378 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:24.022379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:24.022399 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:24.022465 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:24.022988 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:24.023620 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:24.024835 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:24.025065 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:24.025137 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:24.028577 (+ 3440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.3:48453: BlockingRecv error: recv got EOF from 127.13.207.3:48453 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 07:02:24 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:24 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:24.771924 23681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:24.765560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.1:56571)
0808 07:02:24.765736 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:24.765747 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:24.765803 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:24.766129 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:24.766130 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:24.766307 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:24.766362 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:24.766363 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:24.767180 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:24.767182 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:24.767803 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:24.767805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:24.768677 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:24.768679 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:24.768757 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:24.769480 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:24.769486 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:24.771859 (+ 2373us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":101,"threads_started":1}
W20260808 07:02:24.772225 23682 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:24.765707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:56571 (local address 127.13.207.62:38637)
0808 07:02:24.765942 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:24.765944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:24.765954 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:02:24.765984 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:24.765984 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:24.765997 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:24.766085 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:24.766482 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:24.767075 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:24.767960 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:24.768490 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:24.768569 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:24.772149 (+ 3580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.1:56571: BlockingRecv error: recv got EOF from 127.13.207.1:56571 (error 108)
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":112,"threads_started":1}
Aug 08 07:02:24 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:24 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:24.889909 23683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:24.884066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.2:51461)
0808 07:02:24.884310 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:24.884324 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:24.884412 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:24.884616 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:24.884617 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:24.884845 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:24.884897 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:24.884898 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:24.885742 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:24.885743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:24.886460 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:24.886463 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:24.887016 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:24.887017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:24.887067 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:24.887622 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:24.887627 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:24.889812 (+ 2185us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":90,"threads_started":1}
W20260808 07:02:24.890019 23682 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:24.884120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:51461 (local address 127.13.207.62:38637)
0808 07:02:24.884218 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:24.884220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:24.884410 (+ 190us) server_negotiation.cc:407] Connection header received
0808 07:02:24.884511 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:24.884511 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:24.884527 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:24.884598 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:24.885033 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:24.885649 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:24.886575 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:24.886895 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:24.886981 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:24.889964 (+ 2983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.2:51461: BlockingRecv error: recv got EOF from 127.13.207.2:51461 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 07:02:25 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:25 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:25.035542 23684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:25.029132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.3:35183)
0808 07:02:25.029387 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:25.029398 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:25.029456 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:25.029731 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:25.029733 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:25.029971 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:25.030032 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:25.030034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:25.030982 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:25.030983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:25.031672 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:25.031676 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:25.032214 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:25.032215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:25.032286 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:25.032916 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:25.032928 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:25.035442 (+ 2514us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":186,"threads_started":1}
W20260808 07:02:25.035755 23682 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:25.029407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:35183 (local address 127.13.207.62:38637)
0808 07:02:25.029498 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:25.029500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:25.029513 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:02:25.029542 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:25.029543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:25.029557 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:25.029625 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:25.030180 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:25.030745 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:25.031871 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:25.032084 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:25.032194 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:25.035688 (+ 3494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.3:35183: BlockingRecv error: recv got EOF from 127.13.207.3:35183 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 07:02:25 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:25 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:25.778638 23685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:25.772669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.1:49503)
0808 07:02:25.772855 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:25.772868 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:25.772935 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:25.773125 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:25.773126 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:25.773333 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:25.773437 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:25.773440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:25.774276 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:25.774277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:25.774886 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:25.774889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:25.775383 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:25.775384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:25.775437 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:25.776093 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:25.776099 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:25.778499 (+ 2400us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":102,"threads_started":1}
W20260808 07:02:25.778784 23686 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:25.772800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:49503 (local address 127.13.207.62:38637)
0808 07:02:25.772952 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:25.772954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:25.772964 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:02:25.772992 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:25.772993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:25.773004 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:25.773082 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:25.773554 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:25.774138 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:25.775020 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:25.775252 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:25.775328 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:25.778655 (+ 3327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.1:49503: BlockingRecv error: recv got EOF from 127.13.207.1:49503 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":80,"threads_started":1}
Aug 08 07:02:25 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:25 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:25.897429 23687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:25.890815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.2:39023)
0808 07:02:25.891089 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:25.891106 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:25.891227 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:25.891453 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:25.891454 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:25.891645 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:25.891692 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:25.891694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:25.892506 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:25.892507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:25.893673 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:25.893677 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:25.894079 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:25.894081 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:25.894285 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:25.895047 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:25.895052 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:25.897348 (+ 2296us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":85,"threads_started":1}
W20260808 07:02:25.897640 23686 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:25.890927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:39023 (local address 127.13.207.62:38637)
0808 07:02:25.891123 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:25.891126 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:25.891197 (+ 71us) server_negotiation.cc:407] Connection header received
0808 07:02:25.891308 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:25.891309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:25.891330 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:25.891386 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:25.891776 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:25.892323 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:25.893755 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:25.893963 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:25.894102 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:25.897561 (+ 3459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.2:39023: BlockingRecv error: recv got EOF from 127.13.207.2:39023 (error 108)
Metrics: {"server-negotiator.queue_time_us":166}
Aug 08 07:02:26 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:26 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:26.041388 23688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:26.036115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.3:35649)
0808 07:02:26.036289 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:26.036300 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:26.036357 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:26.036505 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:26.036505 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:26.036681 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:26.036776 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:26.036778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:26.037505 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:26.037506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:26.038105 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:26.038107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:26.038447 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:26.038448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:26.038487 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:26.038952 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:26.038957 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:26.041259 (+ 2302us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":113,"threads_started":1}
W20260808 07:02:26.041451 23686 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:26.036239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:35649 (local address 127.13.207.62:38637)
0808 07:02:26.036307 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:26.036309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:26.036343 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:02:26.036409 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:26.036410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:26.036423 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:26.036484 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:26.036860 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:26.037419 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:26.038180 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:26.038360 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:26.038439 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:26.041371 (+ 2932us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.3:35649: BlockingRecv error: recv got EOF from 127.13.207.3:35649 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 08 07:02:26 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:26 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:26.785024 23689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:26.779298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.1:37871)
0808 07:02:26.779467 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:26.779483 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:26.779569 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:26.779814 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:26.779815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:26.780103 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:26.780174 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:26.780177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:26.780906 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:26.780949 (+ 43us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:26.781589 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:26.781592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:26.781963 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:26.781964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:26.782008 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:26.782546 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:26.782552 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:26.784922 (+ 2370us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":101,"threads_started":1}
W20260808 07:02:26.785130 23691 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:26.779330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:37871 (local address 127.13.207.62:38637)
0808 07:02:26.779484 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:26.779486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:26.779632 (+ 146us) server_negotiation.cc:407] Connection header received
0808 07:02:26.779671 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:26.779672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:26.779690 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:26.779780 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:26.780275 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:26.780803 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:26.781683 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:26.781886 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:26.781945 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:26.785044 (+ 3099us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.1:37871: BlockingRecv error: recv got EOF from 127.13.207.1:37871 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
Aug 08 07:02:26 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:26 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:26.904017 23692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:26.898066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.2:59365)
0808 07:02:26.898231 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:26.898246 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:26.898327 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:26.898624 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:26.898625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:26.898869 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:26.898920 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:26.898921 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:26.899725 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:26.899727 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:26.900360 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:26.900362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:26.900793 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:26.900795 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:26.900865 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:26.901407 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:26.901413 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:26.903900 (+ 2487us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":95,"threads_started":1}
W20260808 07:02:26.904080 23691 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:26.898204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:59365 (local address 127.13.207.62:38637)
0808 07:02:26.898296 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:26.898299 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:26.898312 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:02:26.898448 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:26.898449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:26.898463 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:26.898607 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:26.899018 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:26.899612 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:26.900465 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:26.900704 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:26.900796 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:26.904007 (+ 3211us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.2:59365: BlockingRecv error: recv got EOF from 127.13.207.2:59365 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 07:02:27 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:27 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:27.047739 23693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:27.042054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.3:41413)
0808 07:02:27.042328 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:27.042345 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:27.042455 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:27.042630 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:27.042631 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:27.042939 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:27.042993 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:27.042996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:27.043805 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:27.043806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:27.044410 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:27.044413 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:27.044833 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:27.044834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:27.044910 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:27.045494 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:27.045499 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:27.047644 (+ 2145us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":112,"threads_started":1}
W20260808 07:02:27.047820 23691 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:27.042206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:41413 (local address 127.13.207.62:38637)
0808 07:02:27.042361 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:27.042364 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:27.042447 (+ 83us) server_negotiation.cc:407] Connection header received
0808 07:02:27.042524 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:27.042525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:27.042547 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:27.042622 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:27.043134 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:27.043692 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:27.044520 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:27.044735 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:27.044827 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:27.047751 (+ 2924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.3:41413: BlockingRecv error: recv got EOF from 127.13.207.3:41413 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 08 07:02:27 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:27 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:27.796065 23694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:27.785748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.1:38277)
0808 07:02:27.785990 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:27.786003 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:27.786097 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:27.786398 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:27.786399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:27.786709 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:27.786777 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:27.786780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:27.787642 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:27.787643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:27.788250 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:27.788253 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:27.788710 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:27.788711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:27.788767 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:27.793680 (+ 4913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:27.793688 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:27.795978 (+ 2290us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":139,"threads_started":1}
W20260808 07:02:27.796190 23695 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:27.785783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:38277 (local address 127.13.207.62:38637)
0808 07:02:27.785990 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:27.785993 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:27.786082 (+ 89us) server_negotiation.cc:407] Connection header received
0808 07:02:27.786254 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:27.786255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:27.786273 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:27.786353 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:27.786864 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:27.787532 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:27.788384 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:27.788603 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:27.788677 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:27.796111 (+ 7434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.1:38277: BlockingRecv error: recv got EOF from 127.13.207.1:38277 (error 108)
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":101,"threads_started":1}
Aug 08 07:02:27 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:27 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:27.910043 23696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:27.904727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.2:59979)
0808 07:02:27.904940 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:27.904957 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:27.905042 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:27.905220 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:27.905221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:27.905396 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:27.905481 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:27.905483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:27.906200 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:27.906201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:27.906807 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:27.906809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:27.907277 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:27.907278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:27.907332 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:27.907810 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:27.907814 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:27.909906 (+ 2092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":100,"threads_started":1}
W20260808 07:02:27.910410 23695 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:27.904920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:59979 (local address 127.13.207.62:38637)
0808 07:02:27.904996 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:27.904997 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:27.905013 (+ 16us) server_negotiation.cc:407] Connection header received
0808 07:02:27.905127 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:27.905128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:27.905141 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:27.905193 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:27.905580 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:27.906097 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:27.906913 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:27.907179 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:27.907250 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:27.910332 (+ 3082us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.2:59979: BlockingRecv error: recv got EOF from 127.13.207.2:59979 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 07:02:28 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:28 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:28.054791 23697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:28.048475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.3:57829)
0808 07:02:28.048624 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:28.048643 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:28.048931 (+ 288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:28.049195 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:28.049196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:28.049496 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:28.049558 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:28.049560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:28.050312 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:28.050313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:28.051089 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:28.051093 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:28.051502 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:28.051503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:28.051562 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:28.052196 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:28.052201 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:28.054641 (+ 2440us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
W20260808 07:02:28.054968 23695 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:28.048629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:57829 (local address 127.13.207.62:38637)
0808 07:02:28.048758 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:28.048761 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:28.048779 (+ 18us) server_negotiation.cc:407] Connection header received
0808 07:02:28.049084 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:28.049085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:28.049104 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:28.049183 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:28.049675 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:28.050210 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:28.051208 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:28.051418 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:28.051503 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:28.054875 (+ 3372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.3:57829: BlockingRecv error: recv got EOF from 127.13.207.3:57829 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 08 07:02:28 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:28 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:28.802611 23698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:28.796786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.1:46167)
0808 07:02:28.796982 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:28.796997 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:28.797109 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:28.797364 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:28.797365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:28.797552 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:28.797608 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:28.797609 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:28.798396 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:28.798397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:28.799153 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:28.799156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:28.799563 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:28.799564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:28.799627 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:28.800164 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:28.800169 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:28.802508 (+ 2339us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":78,"threads_started":1}
W20260808 07:02:28.802744 23699 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:28.796924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:46167 (local address 127.13.207.62:38637)
0808 07:02:28.797185 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:28.797187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:28.797198 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:02:28.797227 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:28.797227 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:28.797238 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:28.797311 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:28.797693 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:28.798266 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:28.799246 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:28.799468 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:28.799562 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:28.802637 (+ 3075us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.1:46167: BlockingRecv error: recv got EOF from 127.13.207.1:46167 (error 108)
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":86,"threads_started":1}
Aug 08 07:02:28 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:28 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:28.916914 23700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:28.910955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.2:57467)
0808 07:02:28.911151 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:28.911166 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:28.911258 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:28.911483 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:28.911484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:28.911667 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:28.911728 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:28.911730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:28.912446 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:28.912447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:28.913298 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:28.913301 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:28.913815 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:28.913816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:28.913895 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:28.914749 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:28.914755 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:28.916804 (+ 2049us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":86,"threads_started":1}
W20260808 07:02:28.916983 23699 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:28.911119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:57467 (local address 127.13.207.62:38637)
0808 07:02:28.911199 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:28.911201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:28.911233 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:02:28.911326 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:28.911327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:28.911340 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:28.911393 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:28.911801 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:28.912344 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:28.913433 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:28.913737 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:28.913813 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:28.916920 (+ 3107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.2:57467: BlockingRecv error: recv got EOF from 127.13.207.2:57467 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 07:02:29 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:29 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.3@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:29.062605 23701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:29.055425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.3:47459)
0808 07:02:29.055590 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:29.055602 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:29.055677 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:29.055925 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:29.055926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:29.056128 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:29.056312 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:29.056319 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:29.057505 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:29.057506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:29.058290 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:29.058295 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:29.058835 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:29.058836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:29.058894 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:29.059613 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:29.059619 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:29.062536 (+ 2917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":109,"threads_started":1}
W20260808 07:02:29.062942 23699 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:29.055492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:47459 (local address 127.13.207.62:38637)
0808 07:02:29.055609 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:29.055611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:29.055723 (+ 112us) server_negotiation.cc:407] Connection header received
0808 07:02:29.055757 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:29.055758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:29.055774 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:29.055844 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:29.056779 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:29.057369 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:29.058425 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:29.058637 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:29.058728 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:29.062868 (+ 4140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.3:47459: BlockingRecv error: recv got EOF from 127.13.207.3:47459 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 08 07:02:29 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:29 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.1@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:29.809154 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:29.803292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.1:38947)
0808 07:02:29.803490 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:29.803501 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:29.803574 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:29.803731 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:29.803732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:29.803906 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:29.803957 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:29.803958 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:29.804782 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:29.804783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:29.805601 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:29.805605 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:29.806083 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:29.806084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:29.806145 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:29.806749 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:29.806754 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:29.809051 (+ 2297us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":119,"threads_started":1}
W20260808 07:02:29.809269 23703 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:29.803374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:38947 (local address 127.13.207.62:38637)
0808 07:02:29.803493 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:29.803495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:29.803577 (+ 82us) server_negotiation.cc:407] Connection header received
0808 07:02:29.803626 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:29.803627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:29.803638 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:29.803697 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:29.804028 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:29.804663 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:29.805749 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:29.805958 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:29.806034 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:29.809188 (+ 3154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.1:38947: BlockingRecv error: recv got EOF from 127.13.207.1:38947 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":53,"threads_started":1}
Aug 08 07:02:29 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:02:29 dist-test-slave-8tdl krb5kdc[22592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.207.2@KRBTEST.COM for oryx/127.13.207.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:02:29.924160 23704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:29.917768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38637 (local address 127.13.207.2:49537)
0808 07:02:29.918009 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:29.918023 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:29.918109 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:29.918403 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:29.918404 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:29.918733 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:29.918788 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:29.918790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:29.919625 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:29.919627 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:29.920303 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:29.920306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:29.920816 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:29.920818 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:29.920872 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:29.921409 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:29.921414 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:29.924007 (+ 2593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.207.62:38637: Server oryx/127.13.207.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":137,"threads_started":1}
W20260808 07:02:29.924196 23703 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:29.917951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:49537 (local address 127.13.207.62:38637)
0808 07:02:29.918047 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:29.918049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:29.918197 (+ 148us) server_negotiation.cc:407] Connection header received
0808 07:02:29.918238 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:29.918239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:29.918252 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:29.918322 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:29.918899 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:29.919498 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:29.920418 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:29.920668 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:29.920740 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:29.924130 (+ 3390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.207.2:49537: BlockingRecv error: recv got EOF from 127.13.207.2:49537 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 07:02:29.949776 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 23204
I20260808 07:02:29.957993 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 23365
I20260808 07:02:29.965201 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 23504
I20260808 07:02:29.971761 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 23130
2026-08-08T07:02:29Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17825 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T07:02:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:02:30Z Disabled control of system clock
I20260808 07:02:30.009320 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:33007
--webserver_interface=127.13.207.62
--webserver_port=0
--builtin_ntp_servers=127.13.207.20:33735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:33007
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 07:02:30.115762 23708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:30.116017 23708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:30.116044 23708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:30.117501 23708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:02:30.117568 23708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:30.117583 23708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:30.117595 23708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:02:30.117609 23708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:02:30.119184 23708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:33735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:33007
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:33007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:30.119442 23708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:30.119712 23708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:30.127710 23716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:30.127629 23708 server_base.cc:1034] running on GCE node
W20260808 07:02:30.127635 23714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:30.127658 23713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:30.128379 23708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:30.128774 23708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:30.129968 23708 hybrid_clock.cc:648] HybridClock initialized: now 1786172550129953 us; error 96 us; skew 500 ppm
I20260808 07:02:30.131910 23708 webserver.cc:460] Webserver started at http://127.13.207.62:33751/ using document root <none> and password file <none>
I20260808 07:02:30.132670 23708 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:30.133042 23708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:30.133262 23708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:30.135201 23708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "8342f18afb164d55912270c1b986281c"
format_stamp: "Formatted at 2026-08-08 07:02:30 on dist-test-slave-8tdl"
server_key: "122fbb82ae37e1342ce167f5f28d3927"
server_key_iv: "7ac0e7b7ded5472d926e185569fc0e2a"
server_key_version: "encryptionkey@0"
I20260808 07:02:30.136034 23708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "8342f18afb164d55912270c1b986281c"
format_stamp: "Formatted at 2026-08-08 07:02:30 on dist-test-slave-8tdl"
server_key: "122fbb82ae37e1342ce167f5f28d3927"
server_key_iv: "7ac0e7b7ded5472d926e185569fc0e2a"
server_key_version: "encryptionkey@0"
I20260808 07:02:30.138304 23708 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 07:02:30.140013 23722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:30.140321 23708 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:02:30.140461 23708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "8342f18afb164d55912270c1b986281c"
format_stamp: "Formatted at 2026-08-08 07:02:30 on dist-test-slave-8tdl"
server_key: "122fbb82ae37e1342ce167f5f28d3927"
server_key_iv: "7ac0e7b7ded5472d926e185569fc0e2a"
server_key_version: "encryptionkey@0"
I20260808 07:02:30.140573 23708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:30.166916 23708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:30.167356 23708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:30.167601 23708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:30.173375 23708 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:33007
I20260808 07:02:30.173413 23776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:33007 every 8 connection(s)
I20260808 07:02:30.173950 23708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:02:30.174789 23777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:30.177884 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 23708
I20260808 07:02:30.177970 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:02:30.178118 14140 external_mini_cluster.cc:1428] Setting key 380591a8841dcb1e06cb4ddfd8a7130d
I20260808 07:02:30.179375 23777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8342f18afb164d55912270c1b986281c: Bootstrap starting.
I20260808 07:02:30.180209 23777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8342f18afb164d55912270c1b986281c: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:30.180599 23777 log.cc:824] T 00000000000000000000000000000000 P 8342f18afb164d55912270c1b986281c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:30.181490 23777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8342f18afb164d55912270c1b986281c: No bootstrap required, opened a new log
I20260808 07:02:30.184068 23777 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8342f18afb164d55912270c1b986281c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8342f18afb164d55912270c1b986281c" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33007 } }
I20260808 07:02:30.184317 23777 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8342f18afb164d55912270c1b986281c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:30.184386 23777 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8342f18afb164d55912270c1b986281c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8342f18afb164d55912270c1b986281c, State: Initialized, Role: FOLLOWER
I20260808 07:02:30.184621 23777 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8342f18afb164d55912270c1b986281c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8342f18afb164d55912270c1b986281c" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33007 } }
I20260808 07:02:30.184767 23777 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8342f18afb164d55912270c1b986281c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:02:30.184839 23777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8342f18afb164d55912270c1b986281c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:02:30.184904 23777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8342f18afb164d55912270c1b986281c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:30.185824 23777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8342f18afb164d55912270c1b986281c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8342f18afb164d55912270c1b986281c" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33007 } }
I20260808 07:02:30.185943 23777 leader_election.cc:302] T 00000000000000000000000000000000 P 8342f18afb164d55912270c1b986281c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8342f18afb164d55912270c1b986281c; no voters:
I20260808 07:02:30.186229 23777 leader_election.cc:288] T 00000000000000000000000000000000 P 8342f18afb164d55912270c1b986281c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:02:30.186425 23782 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8342f18afb164d55912270c1b986281c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:02:30.186728 23780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:30.178798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45494 (local address 127.13.207.62:33007)
0808 07:02:30.179279 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:30.179293 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:30.179541 (+ 248us) server_negotiation.cc:407] Connection header received
0808 07:02:30.180061 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:30.180069 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:30.180139 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:30.180423 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:30.180723 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:30.182669 (+ 1946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:30.183713 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:30.184519 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:30.184669 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:30.185648 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:30.185659 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:30.185660 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:02:30.185842 (+ 182us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:30.185878 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:02:30.185891 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:30.185987 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:30.186340 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 07:02:30.186452 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":188,"threads_started":1}
I20260808 07:02:30.186721 23777 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8342f18afb164d55912270c1b986281c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:02:30.186720 23782 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8342f18afb164d55912270c1b986281c [term 1 LEADER]: Becoming Leader. State: Replica: 8342f18afb164d55912270c1b986281c, State: Running, Role: LEADER
I20260808 07:02:30.187124 23782 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8342f18afb164d55912270c1b986281c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8342f18afb164d55912270c1b986281c" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33007 } }
I20260808 07:02:30.187748 23784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8342f18afb164d55912270c1b986281c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8342f18afb164d55912270c1b986281c. Latest consensus state: current_term: 1 leader_uuid: "8342f18afb164d55912270c1b986281c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8342f18afb164d55912270c1b986281c" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33007 } } }
I20260808 07:02:30.188710 23783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8342f18afb164d55912270c1b986281c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8342f18afb164d55912270c1b986281c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8342f18afb164d55912270c1b986281c" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33007 } } }
I20260808 07:02:30.188835 23784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8342f18afb164d55912270c1b986281c [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:30.188954 23783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8342f18afb164d55912270c1b986281c [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:30.189280 23797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:02:30.190956 23797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:02:30.194051 23797 catalog_manager.cc:1347] Generated new cluster ID: a9a3a615429e4fd68e374a53483437e0
I20260808 07:02:30.194159 23797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:02:30.206362 23797 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:02:30.207094 23797 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:02:30.215831 23797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8342f18afb164d55912270c1b986281c: Generated new TSK 0
I20260808 07:02:30.216115 23797 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:02:30.222386 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:33007
--builtin_ntp_servers=127.13.207.20:33735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 07:02:30.324127 23801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:30.324455 23801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:30.324501 23801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:30.326043 23801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:30.326139 23801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:30.326195 23801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:02:30.328055 23801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:33735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:33007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:30.328384 23801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:30.328752 23801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:30.336205 23806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:30.336225 23807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:30.336223 23809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:30.336697 23801 server_base.cc:1034] running on GCE node
I20260808 07:02:30.336944 23801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:30.337291 23801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:30.338518 23801 hybrid_clock.cc:648] HybridClock initialized: now 1786172550338494 us; error 43 us; skew 500 ppm
I20260808 07:02:30.340278 23801 webserver.cc:460] Webserver started at http://127.13.207.1:36273/ using document root <none> and password file <none>
I20260808 07:02:30.340598 23801 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:30.340675 23801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:30.340842 23801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:30.342212 23801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "4450a1c2f0a94d8f81d356e8bf05d34e"
format_stamp: "Formatted at 2026-08-08 07:02:30 on dist-test-slave-8tdl"
server_key: "764048e23a6893a51988000a35dabced"
server_key_iv: "45a1e5b20ff1c70187bcdfe0dd8f79a4"
server_key_version: "encryptionkey@0"
I20260808 07:02:30.342850 23801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "4450a1c2f0a94d8f81d356e8bf05d34e"
format_stamp: "Formatted at 2026-08-08 07:02:30 on dist-test-slave-8tdl"
server_key: "764048e23a6893a51988000a35dabced"
server_key_iv: "45a1e5b20ff1c70187bcdfe0dd8f79a4"
server_key_version: "encryptionkey@0"
I20260808 07:02:30.345247 23801 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 07:02:30.346132 23815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:30.346319 23801 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:30.346441 23801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "4450a1c2f0a94d8f81d356e8bf05d34e"
format_stamp: "Formatted at 2026-08-08 07:02:30 on dist-test-slave-8tdl"
server_key: "764048e23a6893a51988000a35dabced"
server_key_iv: "45a1e5b20ff1c70187bcdfe0dd8f79a4"
server_key_version: "encryptionkey@0"
I20260808 07:02:30.346529 23801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:30.369932 23801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:30.370329 23801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:30.370507 23801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:30.370852 23801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:30.371241 23801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:30.371281 23801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:30.371306 23801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:30.371345 23801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:30.378559 23801 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:40231
I20260808 07:02:30.378608 23932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:40231 every 8 connection(s)
I20260808 07:02:30.379012 23801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:02:30.380143 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 23801
I20260808 07:02:30.380226 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:02:30.380461 14140 external_mini_cluster.cc:1428] Setting key 5c6a62c81042b98f33a22a201ff096c7
I20260808 07:02:30.381649 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:33007
--builtin_ntp_servers=127.13.207.20:33735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 07:02:30.386018 23780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:30.379826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:46863 (local address 127.13.207.62:33007)
0808 07:02:30.379896 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:30.379899 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:30.380269 (+ 370us) server_negotiation.cc:407] Connection header received
0808 07:02:30.381041 (+ 772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:30.381042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:30.381057 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:30.381207 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:30.382120 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:30.382907 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:30.384163 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:30.384480 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:30.384557 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:30.385590 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:30.385591 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:30.385591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:02:30.385597 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:30.385611 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:02:30.385616 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:30.385671 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:30.385812 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 07:02:30.385949 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:02:30.386196 23935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:30.379830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:33007 (local address 127.13.207.1:46863)
0808 07:02:30.380096 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:30.380140 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:30.380906 (+ 766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:30.381316 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:30.381321 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:30.381332 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:30.382010 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:30.382023 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:30.383081 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:30.383082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:30.384062 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:30.384065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:30.384609 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:30.384610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:30.384783 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:30.385405 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:02:30.385406 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:30.385448 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:02:30.385480 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:30.385689 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:30.385695 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:30.385698 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:02:30.385771 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 07:02:30.385894 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":102,"threads_started":1}
I20260808 07:02:30.386893 23933 heartbeater.cc:347] Connected to a master server at 127.13.207.62:33007
I20260808 07:02:30.387017 23933 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:30.387272 23933 heartbeater.cc:511] Master 127.13.207.62:33007 requested a full tablet report, sending...
I20260808 07:02:30.387976 23740 ts_manager.cc:194] Registered new tserver with Master: 4450a1c2f0a94d8f81d356e8bf05d34e (127.13.207.1:40231)
I20260808 07:02:30.388799 23740 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.207.1:46863
W20260808 07:02:30.479121 23936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:30.479409 23936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:30.479452 23936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:30.480888 23936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:30.480983 23936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:30.481038 23936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:02:30.482749 23936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:33735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:33007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:30.483105 23936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:30.483480 23936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:02:30.491122 23936 server_base.cc:1034] running on GCE node
W20260808 07:02:30.491151 23944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:30.491078 23942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:30.491099 23941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:30.491994 23936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:30.492339 23936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:30.493563 23936 hybrid_clock.cc:648] HybridClock initialized: now 1786172550493520 us; error 58 us; skew 500 ppm
I20260808 07:02:30.495373 23936 webserver.cc:460] Webserver started at http://127.13.207.2:40151/ using document root <none> and password file <none>
I20260808 07:02:30.495654 23936 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:30.495710 23936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:30.495843 23936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:30.497023 23936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "e6836e74f97f45acaa1346f3ac0884c9"
format_stamp: "Formatted at 2026-08-08 07:02:30 on dist-test-slave-8tdl"
server_key: "be81016a18e2df468865b4461f6c5c7b"
server_key_iv: "4949c0f628cdb7b7a31fe6f67b8685a1"
server_key_version: "encryptionkey@0"
I20260808 07:02:30.497409 23936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "e6836e74f97f45acaa1346f3ac0884c9"
format_stamp: "Formatted at 2026-08-08 07:02:30 on dist-test-slave-8tdl"
server_key: "be81016a18e2df468865b4461f6c5c7b"
server_key_iv: "4949c0f628cdb7b7a31fe6f67b8685a1"
server_key_version: "encryptionkey@0"
I20260808 07:02:30.499473 23936 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 07:02:30.500700 23950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:30.501278 23936 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260808 07:02:30.501418 23936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "e6836e74f97f45acaa1346f3ac0884c9"
format_stamp: "Formatted at 2026-08-08 07:02:30 on dist-test-slave-8tdl"
server_key: "be81016a18e2df468865b4461f6c5c7b"
server_key_iv: "4949c0f628cdb7b7a31fe6f67b8685a1"
server_key_version: "encryptionkey@0"
I20260808 07:02:30.501669 23936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:30.518427 23936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:30.518890 23936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:30.519047 23936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:30.519320 23936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:30.519678 23936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:30.519732 23936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:30.519778 23936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:30.519805 23936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:30.527199 23936 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:33055
I20260808 07:02:30.527251 24067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:33055 every 8 connection(s)
I20260808 07:02:30.527629 23936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:02:30.529623 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 23936
I20260808 07:02:30.529735 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:02:30.529960 14140 external_mini_cluster.cc:1428] Setting key 94ab2b4032c8f56ca24f9e6c35467651
I20260808 07:02:30.531107 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:33007
--builtin_ntp_servers=127.13.207.20:33735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 07:02:30.534255 23780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:30.528599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:43465 (local address 127.13.207.62:33007)
0808 07:02:30.528696 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:30.528698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:30.528836 (+ 138us) server_negotiation.cc:407] Connection header received
0808 07:02:30.529498 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:30.529499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:30.529510 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:30.529550 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:30.530622 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:30.531333 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:30.532512 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:30.532807 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:30.532893 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:30.533885 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:30.533887 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:30.533898 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:02:30.533905 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:30.533923 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:02:30.533929 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:30.533987 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:30.534120 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 07:02:30.534159 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 07:02:30.534535 24070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:30.528489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:33007 (local address 127.13.207.2:43465)
0808 07:02:30.528708 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:30.528747 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:30.529348 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:30.529652 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:30.529657 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:30.529669 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:30.530516 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:30.530523 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:30.531473 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:30.531474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:30.532381 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:30.532383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:30.532906 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:30.532915 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:30.533075 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:30.533711 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:02:30.533712 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:30.533747 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:02:30.533774 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:30.534023 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:30.534029 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:30.534031 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:02:30.534122 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 07:02:30.534254 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
I20260808 07:02:30.535429 24068 heartbeater.cc:347] Connected to a master server at 127.13.207.62:33007
I20260808 07:02:30.535535 24068 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:30.535785 24068 heartbeater.cc:511] Master 127.13.207.62:33007 requested a full tablet report, sending...
I20260808 07:02:30.536302 23740 ts_manager.cc:194] Registered new tserver with Master: e6836e74f97f45acaa1346f3ac0884c9 (127.13.207.2:33055)
I20260808 07:02:30.536715 23740 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.207.2:43465
W20260808 07:02:30.625032 24071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:30.625298 24071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:30.625339 24071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:30.626648 24071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:30.626811 24071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:30.626860 24071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:02:30.628391 24071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:33735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:33007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:30.628729 24071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:30.629097 24071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:30.634902 24076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:30.635126 24071 server_base.cc:1034] running on GCE node
W20260808 07:02:30.635109 24077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:30.634915 24079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:30.635514 24071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:30.635842 24071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:30.637045 24071 hybrid_clock.cc:648] HybridClock initialized: now 1786172550637022 us; error 39 us; skew 500 ppm
I20260808 07:02:30.638980 24071 webserver.cc:460] Webserver started at http://127.13.207.3:37131/ using document root <none> and password file <none>
I20260808 07:02:30.639297 24071 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:30.639379 24071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:30.639559 24071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:30.640899 24071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "7fb65618db05414188361b4d1fc1d220"
format_stamp: "Formatted at 2026-08-08 07:02:30 on dist-test-slave-8tdl"
server_key: "51163a52b20660b2c8c401a66f9d42aa"
server_key_iv: "c08af5fb0ebc46eeb6c5aa29b6b0355c"
server_key_version: "encryptionkey@0"
I20260808 07:02:30.641301 24071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "7fb65618db05414188361b4d1fc1d220"
format_stamp: "Formatted at 2026-08-08 07:02:30 on dist-test-slave-8tdl"
server_key: "51163a52b20660b2c8c401a66f9d42aa"
server_key_iv: "c08af5fb0ebc46eeb6c5aa29b6b0355c"
server_key_version: "encryptionkey@0"
I20260808 07:02:30.642999 24071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 07:02:30.644070 24085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:30.644356 24071 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:30.644508 24071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "7fb65618db05414188361b4d1fc1d220"
format_stamp: "Formatted at 2026-08-08 07:02:30 on dist-test-slave-8tdl"
server_key: "51163a52b20660b2c8c401a66f9d42aa"
server_key_iv: "c08af5fb0ebc46eeb6c5aa29b6b0355c"
server_key_version: "encryptionkey@0"
I20260808 07:02:30.644603 24071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:30.664578 24071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:30.664968 24071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:30.665145 24071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:30.665405 24071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:30.665766 24071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:30.665823 24071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:30.665867 24071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:30.665897 24071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:30.672640 24071 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:40015
I20260808 07:02:30.672690 24202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:40015 every 8 connection(s)
I20260808 07:02:30.673074 24071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:02:30.678476 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 24071
I20260808 07:02:30.678584 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:02:30.678891 14140 external_mini_cluster.cc:1428] Setting key 7b3c1078982c4a98e2ee2b8c45b76880
I20260808 07:02:30.679687 23780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:30.674010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:43715 (local address 127.13.207.62:33007)
0808 07:02:30.674260 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:30.674262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:30.674276 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:02:30.674859 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:30.674860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:30.674870 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:30.674920 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:30.675892 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:30.676466 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:30.677651 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:30.677859 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:30.677941 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:30.679271 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:30.679273 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:30.679274 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:02:30.679281 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:30.679299 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:02:30.679305 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:30.679364 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:30.679547 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 07:02:30.679583 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218}
I20260808 07:02:30.680253 24205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:30.673926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:33007 (local address 127.13.207.3:43715)
0808 07:02:30.674150 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:30.674190 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:30.674741 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:30.674991 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:30.674995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:30.675003 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:30.675715 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:30.675722 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:30.676603 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:30.676604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:30.677538 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:30.677541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:30.677972 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:30.677973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:30.678291 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:30.679075 (+ 784us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:02:30.679075 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:30.679126 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:02:30.679153 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:30.679424 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:30.679433 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:30.679436 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:02:30.679532 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 07:02:30.679700 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":85,"threads_started":1}
I20260808 07:02:30.681192 24203 heartbeater.cc:347] Connected to a master server at 127.13.207.62:33007
I20260808 07:02:30.681352 24203 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:30.681615 24203 heartbeater.cc:511] Master 127.13.207.62:33007 requested a full tablet report, sending...
I20260808 07:02:30.682401 23740 ts_manager.cc:194] Registered new tserver with Master: 7fb65618db05414188361b4d1fc1d220 (127.13.207.3:40015)
I20260808 07:02:30.682859 23740 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.207.3:43715
I20260808 07:02:30.691949 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:02:30.694473 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 23801
W20260808 07:02:30.694552 23780 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:30.693728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45498 (local address 127.13.207.62:33007)
0808 07:02:30.693836 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:30.693838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:30.693850 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:02:30.693944 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:30.693944 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:30.693956 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:30.694017 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:30.694334 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45498: BlockingRecv error: recv got EOF from 127.0.0.1:45498 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 07:02:30.701119 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 23936
I20260808 07:02:30.706593 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 24071
I20260808 07:02:30.712271 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 23708
2026-08-08T07:02:30Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (736 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T07:02:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:02:30Z Disabled control of system clock
I20260808 07:02:30.744474 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:38699
--webserver_interface=127.13.207.62
--webserver_port=0
--builtin_ntp_servers=127.13.207.20:40239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:38699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 07:02:30.834306 24214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:30.834568 24214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:30.834594 24214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:30.836180 24214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:02:30.836261 24214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:30.836277 24214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:30.836290 24214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:02:30.836303 24214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:02:30.838558 24214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:40239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:38699
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:38699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:30.838891 24214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:30.839301 24214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:30.845227 24220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:30.845261 24221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:30.845487 24214 server_base.cc:1034] running on GCE node
W20260808 07:02:30.845307 24223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:30.845813 24214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:30.846124 24214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:30.847368 24214 hybrid_clock.cc:648] HybridClock initialized: now 1786172550847280 us; error 105 us; skew 500 ppm
I20260808 07:02:30.849354 24214 webserver.cc:460] Webserver started at http://127.13.207.62:39181/ using document root <none> and password file <none>
I20260808 07:02:30.849653 24214 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:30.849731 24214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:30.849896 24214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:30.851197 24214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "ab2973c43f814f1db4f10448f85d20a6"
format_stamp: "Formatted at 2026-08-08 07:02:30 on dist-test-slave-8tdl"
server_key: "bc515bd96ac920d93a0d81b6daa94d9a"
server_key_iv: "9f971d5a172a4d8891108800836918a3"
server_key_version: "encryptionkey@0"
I20260808 07:02:30.851624 24214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "ab2973c43f814f1db4f10448f85d20a6"
format_stamp: "Formatted at 2026-08-08 07:02:30 on dist-test-slave-8tdl"
server_key: "bc515bd96ac920d93a0d81b6daa94d9a"
server_key_iv: "9f971d5a172a4d8891108800836918a3"
server_key_version: "encryptionkey@0"
I20260808 07:02:30.853145 24214 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:02:30.854224 24229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:30.854527 24214 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:30.854650 24214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "ab2973c43f814f1db4f10448f85d20a6"
format_stamp: "Formatted at 2026-08-08 07:02:30 on dist-test-slave-8tdl"
server_key: "bc515bd96ac920d93a0d81b6daa94d9a"
server_key_iv: "9f971d5a172a4d8891108800836918a3"
server_key_version: "encryptionkey@0"
I20260808 07:02:30.854791 24214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:30.858530 24214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:30.858902 24214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:30.859069 24214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:30.864265 24214 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:38699
I20260808 07:02:30.864629 24281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:38699 every 8 connection(s)
I20260808 07:02:30.864902 24214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:02:30.865828 24282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:30.869168 24282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab2973c43f814f1db4f10448f85d20a6: Bootstrap starting.
I20260808 07:02:30.869899 24282 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab2973c43f814f1db4f10448f85d20a6: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:30.870237 24282 log.cc:824] T 00000000000000000000000000000000 P ab2973c43f814f1db4f10448f85d20a6: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:30.870965 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 24214
I20260808 07:02:30.871073 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:02:30.871285 14140 external_mini_cluster.cc:1428] Setting key 967b71f340e30af31027ab9cf08367b0
I20260808 07:02:30.871377 24282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab2973c43f814f1db4f10448f85d20a6: No bootstrap required, opened a new log
I20260808 07:02:30.874145 24282 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab2973c43f814f1db4f10448f85d20a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2973c43f814f1db4f10448f85d20a6" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38699 } }
I20260808 07:02:30.874418 24282 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab2973c43f814f1db4f10448f85d20a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:30.874478 24282 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab2973c43f814f1db4f10448f85d20a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab2973c43f814f1db4f10448f85d20a6, State: Initialized, Role: FOLLOWER
I20260808 07:02:30.874691 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:30.871934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41004 (local address 127.13.207.62:38699)
0808 07:02:30.872190 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:30.872198 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:30.872224 (+ 26us) server_negotiation.cc:407] Connection header received
0808 07:02:30.872877 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:30.872885 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:30.872931 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:30.873136 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:30.874098 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:30.874102 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:30.874102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:02:30.874107 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:30.874152 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:02:30.874164 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:30.874232 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:30.874382 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 07:02:30.874479 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":103,"threads_started":1}
I20260808 07:02:30.874655 24282 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab2973c43f814f1db4f10448f85d20a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2973c43f814f1db4f10448f85d20a6" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38699 } }
I20260808 07:02:30.874910 24282 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab2973c43f814f1db4f10448f85d20a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:02:30.874965 24282 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab2973c43f814f1db4f10448f85d20a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:02:30.875015 24282 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab2973c43f814f1db4f10448f85d20a6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:30.876152 24282 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab2973c43f814f1db4f10448f85d20a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2973c43f814f1db4f10448f85d20a6" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38699 } }
I20260808 07:02:30.876271 24282 leader_election.cc:302] T 00000000000000000000000000000000 P ab2973c43f814f1db4f10448f85d20a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab2973c43f814f1db4f10448f85d20a6; no voters:
I20260808 07:02:30.876484 24282 leader_election.cc:288] T 00000000000000000000000000000000 P ab2973c43f814f1db4f10448f85d20a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:02:30.876523 24287 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab2973c43f814f1db4f10448f85d20a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:02:30.876776 24287 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab2973c43f814f1db4f10448f85d20a6 [term 1 LEADER]: Becoming Leader. State: Replica: ab2973c43f814f1db4f10448f85d20a6, State: Running, Role: LEADER
I20260808 07:02:30.876786 24282 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab2973c43f814f1db4f10448f85d20a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:02:30.876941 24287 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab2973c43f814f1db4f10448f85d20a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2973c43f814f1db4f10448f85d20a6" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38699 } }
I20260808 07:02:30.877488 24288 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab2973c43f814f1db4f10448f85d20a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab2973c43f814f1db4f10448f85d20a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2973c43f814f1db4f10448f85d20a6" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38699 } } }
I20260808 07:02:30.877612 24288 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab2973c43f814f1db4f10448f85d20a6 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:30.877489 24289 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab2973c43f814f1db4f10448f85d20a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab2973c43f814f1db4f10448f85d20a6. Latest consensus state: current_term: 1 leader_uuid: "ab2973c43f814f1db4f10448f85d20a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2973c43f814f1db4f10448f85d20a6" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 38699 } } }
I20260808 07:02:30.877660 24289 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab2973c43f814f1db4f10448f85d20a6 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:30.877912 24295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:02:30.878875 24295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:02:30.881314 24295 catalog_manager.cc:1347] Generated new cluster ID: a02fbc3d8ebb489abe6752c07caa2b4e
I20260808 07:02:30.881395 24295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:02:30.894006 24295 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:02:30.894220 24295 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:02:30.897253 24295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab2973c43f814f1db4f10448f85d20a6: Generated new TSK 0
I20260808 07:02:30.897507 24295 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:02:30.905858 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:38699
--builtin_ntp_servers=127.13.207.20:40239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 07:02:31.013984 24306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:31.014286 24306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:31.014331 24306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:31.015935 24306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:31.016037 24306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:31.016090 24306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:02:31.018416 24306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:40239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:38699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:31.018801 24306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:31.019160 24306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:31.026533 24311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:31.026526 24312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:31.026881 24306 server_base.cc:1034] running on GCE node
W20260808 07:02:31.026526 24314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:31.027274 24306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:31.027581 24306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:31.028803 24306 hybrid_clock.cc:648] HybridClock initialized: now 1786172551028777 us; error 49 us; skew 500 ppm
I20260808 07:02:31.031013 24306 webserver.cc:460] Webserver started at http://127.13.207.1:44483/ using document root <none> and password file <none>
I20260808 07:02:31.031294 24306 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:31.031374 24306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:31.031540 24306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:31.032791 24306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "d3f97f2b420544138bb7e4e58c65f827"
format_stamp: "Formatted at 2026-08-08 07:02:31 on dist-test-slave-8tdl"
server_key: "f4a7e4fc43419910a63e55daef707a4f"
server_key_iv: "5d597ae289e16b15cf23b5ba6b49b1f9"
server_key_version: "encryptionkey@0"
I20260808 07:02:31.033206 24306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "d3f97f2b420544138bb7e4e58c65f827"
format_stamp: "Formatted at 2026-08-08 07:02:31 on dist-test-slave-8tdl"
server_key: "f4a7e4fc43419910a63e55daef707a4f"
server_key_iv: "5d597ae289e16b15cf23b5ba6b49b1f9"
server_key_version: "encryptionkey@0"
I20260808 07:02:31.034960 24306 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 07:02:31.036058 24320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:31.036321 24306 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:31.036446 24306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "d3f97f2b420544138bb7e4e58c65f827"
format_stamp: "Formatted at 2026-08-08 07:02:31 on dist-test-slave-8tdl"
server_key: "f4a7e4fc43419910a63e55daef707a4f"
server_key_iv: "5d597ae289e16b15cf23b5ba6b49b1f9"
server_key_version: "encryptionkey@0"
I20260808 07:02:31.036535 24306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:31.040611 24306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:31.040928 24306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:31.041155 24306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:31.041420 24306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:31.041767 24306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:31.041823 24306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:31.041867 24306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:31.041893 24306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:31.049614 24306 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:34569
I20260808 07:02:31.049649 24436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:34569 every 8 connection(s)
I20260808 07:02:31.050164 24306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:02:31.053282 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 24306
I20260808 07:02:31.053419 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:02:31.053639 14140 external_mini_cluster.cc:1428] Setting key de8dced6696bb33a8c147ff0c55a5065
I20260808 07:02:31.054229 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:31.051328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:48017 (local address 127.13.207.62:38699)
0808 07:02:31.051405 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:31.051408 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:31.051750 (+ 342us) server_negotiation.cc:407] Connection header received
0808 07:02:31.052238 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:31.052239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:31.052245 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:31.052312 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:31.053926 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:31.053928 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:31.053929 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:02:31.053935 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:31.053960 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:02:31.053964 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:31.054034 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:31.054134 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 07:02:31.054155 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:02:31.054481 24439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:31.051273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38699 (local address 127.13.207.1:48017)
0808 07:02:31.051579 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:31.051618 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:31.052113 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:31.052468 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:31.052471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:31.052480 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:31.053720 (+ 1240us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:02:31.053721 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:31.053770 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:02:31.053810 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:31.054034 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:31.054042 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:31.054045 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:02:31.054146 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 07:02:31.054218 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":116,"threads_started":1}
I20260808 07:02:31.054978 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:38699
--builtin_ntp_servers=127.13.207.20:40239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 07:02:31.055153 24437 heartbeater.cc:347] Connected to a master server at 127.13.207.62:38699
I20260808 07:02:31.055260 24437 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:31.055425 24437 heartbeater.cc:511] Master 127.13.207.62:38699 requested a full tablet report, sending...
I20260808 07:02:31.056090 24246 ts_manager.cc:194] Registered new tserver with Master: d3f97f2b420544138bb7e4e58c65f827 (127.13.207.1:34569)
W20260808 07:02:31.162844 24440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:31.163159 24440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:31.163209 24440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:31.164999 24440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:31.165181 24440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:31.165251 24440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:02:31.167117 24440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:40239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:38699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:31.167461 24440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:31.168205 24440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:31.175477 24446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:31.175371 24449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:31.175685 24445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:31.175947 24440 server_base.cc:1034] running on GCE node
I20260808 07:02:31.176184 24440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:31.176522 24440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:31.177775 24440 hybrid_clock.cc:648] HybridClock initialized: now 1786172551177760 us; error 72 us; skew 500 ppm
I20260808 07:02:31.179559 24440 webserver.cc:460] Webserver started at http://127.13.207.2:45175/ using document root <none> and password file <none>
I20260808 07:02:31.179858 24440 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:31.179931 24440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:31.180109 24440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:31.181448 24440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "a9931f3a9def40849e5c576f5e478ec0"
format_stamp: "Formatted at 2026-08-08 07:02:31 on dist-test-slave-8tdl"
server_key: "fd0f6d208c3c144cb57767f4cadb8a29"
server_key_iv: "945dc098d21317da55056c8caa94685f"
server_key_version: "encryptionkey@0"
I20260808 07:02:31.181988 24440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "a9931f3a9def40849e5c576f5e478ec0"
format_stamp: "Formatted at 2026-08-08 07:02:31 on dist-test-slave-8tdl"
server_key: "fd0f6d208c3c144cb57767f4cadb8a29"
server_key_iv: "945dc098d21317da55056c8caa94685f"
server_key_version: "encryptionkey@0"
I20260808 07:02:31.184252 24440 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260808 07:02:31.185547 24455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:31.186162 24440 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:31.186434 24440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "a9931f3a9def40849e5c576f5e478ec0"
format_stamp: "Formatted at 2026-08-08 07:02:31 on dist-test-slave-8tdl"
server_key: "fd0f6d208c3c144cb57767f4cadb8a29"
server_key_iv: "945dc098d21317da55056c8caa94685f"
server_key_version: "encryptionkey@0"
I20260808 07:02:31.186614 24440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:31.191793 24440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:31.192444 24440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:31.192804 24440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:31.193240 24440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:31.193897 24440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:31.194103 24440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:31.194200 24440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:31.194235 24440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:31.202039 24440 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:35065
I20260808 07:02:31.202105 24570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:35065 every 8 connection(s)
I20260808 07:02:31.202526 24440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:02:31.203017 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 24440
I20260808 07:02:31.203249 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:02:31.203517 14140 external_mini_cluster.cc:1428] Setting key d725470aa6163e669f5d4ddee0f1a003
I20260808 07:02:31.204808 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:38699
--builtin_ntp_servers=127.13.207.20:40239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 07:02:31.206890 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:31.203893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:50851 (local address 127.13.207.62:38699)
0808 07:02:31.204008 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:31.204010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:31.204426 (+ 416us) server_negotiation.cc:407] Connection header received
0808 07:02:31.204981 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:31.204983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:31.204992 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:31.205029 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:31.206494 (+ 1465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:31.206497 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:31.206497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:02:31.206503 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:31.206522 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:02:31.206527 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:31.206597 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:31.206760 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 07:02:31.206783 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 07:02:31.207442 24573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:31.204015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38699 (local address 127.13.207.2:50851)
0808 07:02:31.204273 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:31.204326 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:31.204820 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:31.205221 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:31.205225 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:31.205233 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:31.206302 (+ 1069us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:02:31.206302 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:31.206337 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:02:31.206373 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:31.206599 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:31.206605 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:31.206609 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:02:31.207089 (+ 480us) client_negotiation.cc:240] Negotiation successful
0808 07:02:31.207180 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":112,"threads_started":1}
I20260808 07:02:31.208302 24571 heartbeater.cc:347] Connected to a master server at 127.13.207.62:38699
I20260808 07:02:31.208429 24571 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:31.208612 24571 heartbeater.cc:511] Master 127.13.207.62:38699 requested a full tablet report, sending...
I20260808 07:02:31.209551 24246 ts_manager.cc:194] Registered new tserver with Master: a9931f3a9def40849e5c576f5e478ec0 (127.13.207.2:35065)
W20260808 07:02:31.309216 24575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:31.309496 24575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:31.309530 24575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:31.311071 24575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:31.311126 24575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:31.311158 24575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:02:31.312832 24575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:40239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:38699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:31.313145 24575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:31.313519 24575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:31.319336 24580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:31.319339 24581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:31.319339 24583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:31.319430 24575 server_base.cc:1034] running on GCE node
I20260808 07:02:31.319850 24575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:31.320127 24575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:31.321305 24575 hybrid_clock.cc:648] HybridClock initialized: now 1786172551321283 us; error 50 us; skew 500 ppm
I20260808 07:02:31.322825 24575 webserver.cc:460] Webserver started at http://127.13.207.3:44215/ using document root <none> and password file <none>
I20260808 07:02:31.323089 24575 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:31.323143 24575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:31.323285 24575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:31.324501 24575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "717e61a832664604926b1f4759862ef5"
format_stamp: "Formatted at 2026-08-08 07:02:31 on dist-test-slave-8tdl"
server_key: "92b4d9d8b9e59f44cdce1b4ff5468206"
server_key_iv: "d665ca4c37a4f605e3ab32cb69a684a4"
server_key_version: "encryptionkey@0"
I20260808 07:02:31.324893 24575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "717e61a832664604926b1f4759862ef5"
format_stamp: "Formatted at 2026-08-08 07:02:31 on dist-test-slave-8tdl"
server_key: "92b4d9d8b9e59f44cdce1b4ff5468206"
server_key_iv: "d665ca4c37a4f605e3ab32cb69a684a4"
server_key_version: "encryptionkey@0"
I20260808 07:02:31.326408 24575 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:02:31.327559 24589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:31.327798 24575 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:31.327920 24575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "717e61a832664604926b1f4759862ef5"
format_stamp: "Formatted at 2026-08-08 07:02:31 on dist-test-slave-8tdl"
server_key: "92b4d9d8b9e59f44cdce1b4ff5468206"
server_key_iv: "d665ca4c37a4f605e3ab32cb69a684a4"
server_key_version: "encryptionkey@0"
I20260808 07:02:31.328032 24575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:31.332147 24575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:31.332656 24575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:31.332836 24575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:31.333076 24575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:31.333431 24575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:31.333487 24575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:31.333529 24575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:31.333559 24575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:31.339972 24575 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:38285
I20260808 07:02:31.340030 24705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:38285 every 8 connection(s)
I20260808 07:02:31.340387 24575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:02:31.343221 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 24575
I20260808 07:02:31.343300 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:02:31.343565 14140 external_mini_cluster.cc:1428] Setting key b89ef3f293cfb56ee7e43165df6ca82c
I20260808 07:02:31.344389 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:31.341505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:36787 (local address 127.13.207.62:38699)
0808 07:02:31.341593 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:31.341595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:31.341930 (+ 335us) server_negotiation.cc:407] Connection header received
0808 07:02:31.342396 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:31.342397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:31.342403 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:31.342465 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:31.344075 (+ 1610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:31.344077 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:31.344077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:02:31.344082 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:31.344102 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:02:31.344108 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:31.344194 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:31.344295 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 07:02:31.344315 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 07:02:31.344730 24708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:31.341492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:38699 (local address 127.13.207.3:36787)
0808 07:02:31.341724 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:31.341761 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:31.342289 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:31.342541 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:31.342544 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:31.342552 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:31.343854 (+ 1302us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:02:31.343855 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:31.343912 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:02:31.343954 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:31.344210 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:31.344218 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:31.344221 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:02:31.344306 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 07:02:31.344376 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":113,"threads_started":1}
I20260808 07:02:31.345340 24706 heartbeater.cc:347] Connected to a master server at 127.13.207.62:38699
I20260808 07:02:31.345499 24706 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:31.345629 24706 heartbeater.cc:511] Master 127.13.207.62:38699 requested a full tablet report, sending...
I20260808 07:02:31.346125 24246 ts_manager.cc:194] Registered new tserver with Master: 717e61a832664604926b1f4759862ef5 (127.13.207.3:38285)
I20260808 07:02:31.356206 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 07:02:31.359043 24286 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:02:31.358375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41010 (local address 127.13.207.62:38699)
0808 07:02:31.358452 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:31.358454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:31.358540 (+ 86us) server_negotiation.cc:407] Connection header received
0808 07:02:31.358641 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:31.358642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:31.358648 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:31.358786 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:31.358945 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41010: BlockingRecv error: recv got EOF from 127.0.0.1:41010 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:02:31.359658 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 24306
I20260808 07:02:31.370760 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 24440
I20260808 07:02:31.377269 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 24575
I20260808 07:02:31.387405 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 24214
2026-08-08T07:02:31Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (677 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:02:31 dist-test-slave-8tdl krb5kdc[24716](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:02:31 dist-test-slave-8tdl krb5kdc[24716](info): set up 2 sockets
Aug 08 07:02:31 dist-test-slave-8tdl krb5kdc[24716](info): commencing operation
krb5kdc: starting...
W20260808 07:02:32.559841 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:02:32 dist-test-slave-8tdl krb5kdc[24716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:02:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:02:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:02:32.723117 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:40417
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:43107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:40417
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:32.819209 24729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:32.819500 24729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:32.819541 24729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:32.820852 24729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:02:32.820947 24729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:32.820972 24729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:32.821007 24729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:02:32.821036 24729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:02:32.823890 24729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:43107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:40417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:40417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:32.824260 24729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:32.824554 24729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:32.830441 24734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:32.830394 24735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:32.830380 24737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:32.830773 24729 server_base.cc:1034] running on GCE node
I20260808 07:02:32.831089 24729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:32.831449 24729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:32.832664 24729 hybrid_clock.cc:648] HybridClock initialized: now 1786172552832657 us; error 40 us; skew 500 ppm
Aug 08 07:02:32 dist-test-slave-8tdl krb5kdc[24716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172552, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:32.839790 24729 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:02:32.840605 24729 webserver.cc:460] Webserver started at http://127.13.207.62:46619/ using document root <none> and password file <none>
I20260808 07:02:32.840931 24729 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:32.841012 24729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:32.841174 24729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:32.842530 24729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "285ff77506ce4d9ab98c8d72a5713fd4"
format_stamp: "Formatted at 2026-08-08 07:02:32 on dist-test-slave-8tdl"
server_key: "443f50860627ff290ffeca29d7fe3f4c"
server_key_iv: "3b7f0755ee64efe945f96a17d1111f6d"
server_key_version: "encryptionkey@0"
I20260808 07:02:32.843006 24729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "285ff77506ce4d9ab98c8d72a5713fd4"
format_stamp: "Formatted at 2026-08-08 07:02:32 on dist-test-slave-8tdl"
server_key: "443f50860627ff290ffeca29d7fe3f4c"
server_key_iv: "3b7f0755ee64efe945f96a17d1111f6d"
server_key_version: "encryptionkey@0"
I20260808 07:02:32.844734 24729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:02:32.845767 24744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:32.846177 24729 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:32.846344 24729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "285ff77506ce4d9ab98c8d72a5713fd4"
format_stamp: "Formatted at 2026-08-08 07:02:32 on dist-test-slave-8tdl"
server_key: "443f50860627ff290ffeca29d7fe3f4c"
server_key_iv: "3b7f0755ee64efe945f96a17d1111f6d"
server_key_version: "encryptionkey@0"
I20260808 07:02:32.846446 24729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:32.860435 24729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:32.863237 24729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:32.863444 24729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:32.868198 24729 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:40417
I20260808 07:02:32.868288 24796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:40417 every 8 connection(s)
I20260808 07:02:32.868695 24729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:02:32.869994 24798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:32.870329 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 24729
I20260808 07:02:32.870438 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:02:32.870664 14140 external_mini_cluster.cc:1428] Setting key 6e157aac2c0dd50325d4e003fdd41566
I20260808 07:02:32.873881 24798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4: Bootstrap starting.
I20260808 07:02:32.874835 24798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:32.875240 24798 log.cc:824] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:32.876120 24798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4: No bootstrap required, opened a new log
Aug 08 07:02:32 dist-test-slave-8tdl krb5kdc[24716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:02:32.879024 24798 raft_consensus.cc:348] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285ff77506ce4d9ab98c8d72a5713fd4" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40417 } }
I20260808 07:02:32.879194 24798 raft_consensus.cc:374] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:32.879217 24798 raft_consensus.cc:729] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 285ff77506ce4d9ab98c8d72a5713fd4, State: Initialized, Role: FOLLOWER
I20260808 07:02:32.879361 24798 consensus_queue.cc:260] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285ff77506ce4d9ab98c8d72a5713fd4" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40417 } }
I20260808 07:02:32.879456 24798 raft_consensus.cc:388] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:02:32.879489 24798 raft_consensus.cc:482] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:02:32.879540 24798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:32.880404 24798 raft_consensus.cc:504] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285ff77506ce4d9ab98c8d72a5713fd4" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40417 } }
I20260808 07:02:32.880517 24798 leader_election.cc:302] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 285ff77506ce4d9ab98c8d72a5713fd4; no voters:
I20260808 07:02:32.880726 24798 leader_election.cc:288] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:02:32.880861 24803 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:02:32.881048 24803 raft_consensus.cc:686] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4 [term 1 LEADER]: Becoming Leader. State: Replica: 285ff77506ce4d9ab98c8d72a5713fd4, State: Running, Role: LEADER
I20260808 07:02:32.881052 24798 sys_catalog.cc:564] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:02:32.881425 24803 consensus_queue.cc:237] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285ff77506ce4d9ab98c8d72a5713fd4" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40417 } }
I20260808 07:02:32.881982 24804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 285ff77506ce4d9ab98c8d72a5713fd4. Latest consensus state: current_term: 1 leader_uuid: "285ff77506ce4d9ab98c8d72a5713fd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285ff77506ce4d9ab98c8d72a5713fd4" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40417 } } }
I20260808 07:02:32.881982 24805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "285ff77506ce4d9ab98c8d72a5713fd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285ff77506ce4d9ab98c8d72a5713fd4" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40417 } } }
I20260808 07:02:32.882185 24805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:32.882133 24804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4 [sys.catalog]: This master's current role is: LEADER
I20260808 07:02:32.882429 24813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:02:32.884069 24813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:02:32.885725 24801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:32.871410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35858 (local address 127.13.207.62:40417)
0808 07:02:32.871642 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:32.871652 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:32.871690 (+ 38us) server_negotiation.cc:407] Connection header received
0808 07:02:32.872164 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:32.872171 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:32.872232 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:32.872382 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:32.873335 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:32.874889 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:32.875958 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:32.876476 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:32.876580 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:32.880028 (+ 3448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:32.880033 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:32.880033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:32.880049 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:32.882555 (+ 2506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:32.883333 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:32.883344 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:32.883345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:32.883474 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:32.883809 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:32.883810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:32.883810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:32.884471 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:32.885217 (+ 746us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:32.885321 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 07:02:32.885515 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":136,"threads_started":1}
I20260808 07:02:32.886720 24813 catalog_manager.cc:1347] Generated new cluster ID: 209e5d566b5b4590828476a1157190df
I20260808 07:02:32.886785 24813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:02:32.920141 24813 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:02:32.921020 24813 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:02:32.925930 24813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4: Generated new TSK 0
I20260808 07:02:32.926240 24813 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
I20260808 07:02:32.987744 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40417
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:43107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:33.125288 24826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:33.125555 24826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:33.125586 24826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:33.127238 24826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:33.127305 24826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:33.127341 24826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:02:33.129179 24826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:43107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:33.129712 24826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:33.130051 24826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:02:33.137749 24826 server_base.cc:1034] running on GCE node
W20260808 07:02:33.137691 24832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:33.137691 24834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:33.137679 24831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:33.138872 24826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:33.139303 24826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:33.140528 24826 hybrid_clock.cc:648] HybridClock initialized: now 1786172553140483 us; error 69 us; skew 500 ppm
Aug 08 07:02:33 dist-test-slave-8tdl krb5kdc[24716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172553, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:33.145407 24826 init.cc:377] Logged in from keytab as kudu/127.13.207.1@KRBTEST.COM (short username kudu)
I20260808 07:02:33.146049 24826 webserver.cc:460] Webserver started at http://127.13.207.1:37739/ using document root <none> and password file <none>
I20260808 07:02:33.146307 24826 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:33.146369 24826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:33.146544 24826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:33.147876 24826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "30c958bfa5204620b9f9ee6537debe0c"
format_stamp: "Formatted at 2026-08-08 07:02:33 on dist-test-slave-8tdl"
server_key: "b28b73f8922e948c940d57f768f63ff6"
server_key_iv: "3f3e7f501a99172331de05a9680601bf"
server_key_version: "encryptionkey@0"
I20260808 07:02:33.148303 24826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "30c958bfa5204620b9f9ee6537debe0c"
format_stamp: "Formatted at 2026-08-08 07:02:33 on dist-test-slave-8tdl"
server_key: "b28b73f8922e948c940d57f768f63ff6"
server_key_iv: "3f3e7f501a99172331de05a9680601bf"
server_key_version: "encryptionkey@0"
I20260808 07:02:33.150064 24826 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 07:02:33.152160 24841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:33.152518 24826 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:02:33.152667 24826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "30c958bfa5204620b9f9ee6537debe0c"
format_stamp: "Formatted at 2026-08-08 07:02:33 on dist-test-slave-8tdl"
server_key: "b28b73f8922e948c940d57f768f63ff6"
server_key_iv: "3f3e7f501a99172331de05a9680601bf"
server_key_version: "encryptionkey@0"
I20260808 07:02:33.152769 24826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:33.171254 24826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:33.175100 24826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:33.175307 24826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:33.175601 24826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:33.175956 24826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:33.176014 24826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:33.176054 24826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:33.176100 24826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:33.184262 24826 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:36991
I20260808 07:02:33.184434 24955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:36991 every 8 connection(s)
I20260808 07:02:33.184772 24826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:02:33.187717 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 24826
I20260808 07:02:33.187870 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:02:33.188087 14140 external_mini_cluster.cc:1428] Setting key 98a159d2b804bea6be277ddd42dc15dc
Aug 08 07:02:33 dist-test-slave-8tdl krb5kdc[24716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172553, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.207.2@KRBTEST.COM; defaulting to no policy
I20260808 07:02:33.203258 24801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:33.186376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:48933 (local address 127.13.207.62:40417)
0808 07:02:33.186508 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:33.186511 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:33.186700 (+ 189us) server_negotiation.cc:407] Connection header received
0808 07:02:33.187364 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:33.187365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:33.187379 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:33.187439 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:33.189079 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.190011 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.192138 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.192381 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.192678 (+ 297us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:33.198194 (+ 5516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:33.198196 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:33.198197 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:33.198209 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:33.200352 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:33.201278 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:33.201280 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:33.201280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:33.201354 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:33.202304 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:33.202306 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:33.202306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:33.202457 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:33.202837 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:33.203069 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 07:02:33.203179 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 07:02:33.203572 24958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:33.185920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:40417 (local address 127.13.207.1:48933)
0808 07:02:33.186501 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:33.186558 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:33.187246 (+ 688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:33.187525 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:33.187530 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:33.188420 (+ 890us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:33.188860 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:33.188868 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.190162 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.190164 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:33.191685 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:33.191690 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.192716 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.192718 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:33.192866 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:33.193891 (+ 1025us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:33.193904 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:33.198018 (+ 4114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:33.200541 (+ 2523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:33.200554 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:33.200556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:33.200853 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:33.201591 (+ 738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:33.201592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:33.201592 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:33.201749 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:33.202584 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:33.202597 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:33.202857 (+ 260us) client_negotiation.cc:769] Sending connection context
0808 07:02:33.202971 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 07:02:33.203168 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":454,"threads_started":1}
I20260808 07:02:33.204613 24956 heartbeater.cc:347] Connected to a master server at 127.13.207.62:40417
I20260808 07:02:33.205029 24956 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.2@KRBTEST.COM" created.
I20260808 07:02:33.205354 24956 heartbeater.cc:511] Master 127.13.207.62:40417 requested a full tablet report, sending...
I20260808 07:02:33.206079 24761 ts_manager.cc:194] Registered new tserver with Master: 30c958bfa5204620b9f9ee6537debe0c (127.13.207.1:36991)
I20260808 07:02:33.206975 24761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:48933
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
I20260808 07:02:33.227175 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40417
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:43107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:33.314925 24961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:33.315204 24961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:33.315246 24961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:33.317029 24961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:33.317111 24961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:33.317159 24961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:02:33.319993 24961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:43107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:33.320333 24961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:33.320667 24961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:33.328334 24967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:33.328349 24969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:33.328626 24966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:33.328692 24961 server_base.cc:1034] running on GCE node
I20260808 07:02:33.329084 24961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:33.329385 24961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:33.330578 24961 hybrid_clock.cc:648] HybridClock initialized: now 1786172553330563 us; error 28 us; skew 500 ppm
Aug 08 07:02:33 dist-test-slave-8tdl krb5kdc[24716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172553, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:33.333877 24961 init.cc:377] Logged in from keytab as kudu/127.13.207.2@KRBTEST.COM (short username kudu)
I20260808 07:02:33.334463 24961 webserver.cc:460] Webserver started at http://127.13.207.2:44621/ using document root <none> and password file <none>
I20260808 07:02:33.334805 24961 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:33.334897 24961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:33.335053 24961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:33.336192 24961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "2f2e4f772d7a414099f365ab5bcfbdce"
format_stamp: "Formatted at 2026-08-08 07:02:33 on dist-test-slave-8tdl"
server_key: "fab7a94725daff85371fc70eda4c79f4"
server_key_iv: "53f6943afc5062442a9fc23822dc3e8f"
server_key_version: "encryptionkey@0"
I20260808 07:02:33.336591 24961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "2f2e4f772d7a414099f365ab5bcfbdce"
format_stamp: "Formatted at 2026-08-08 07:02:33 on dist-test-slave-8tdl"
server_key: "fab7a94725daff85371fc70eda4c79f4"
server_key_iv: "53f6943afc5062442a9fc23822dc3e8f"
server_key_version: "encryptionkey@0"
I20260808 07:02:33.338172 24961 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:02:33.339324 24977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:33.339768 24961 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:02:33.339999 24961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "2f2e4f772d7a414099f365ab5bcfbdce"
format_stamp: "Formatted at 2026-08-08 07:02:33 on dist-test-slave-8tdl"
server_key: "fab7a94725daff85371fc70eda4c79f4"
server_key_iv: "53f6943afc5062442a9fc23822dc3e8f"
server_key_version: "encryptionkey@0"
I20260808 07:02:33.340114 24961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:33.365864 24961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:33.368536 24961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:33.368731 24961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:33.369016 24961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:33.369377 24961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:33.369525 24961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:33.369656 24961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:33.369688 24961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:33.377406 24961 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:38825
I20260808 07:02:33.377439 25094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:38825 every 8 connection(s)
I20260808 07:02:33.377991 24961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:02:33.384778 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 24961
I20260808 07:02:33.384907 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:02:33.385154 14140 external_mini_cluster.cc:1428] Setting key d09d836d0ff0d5af1d35ed24f06653de
Aug 08 07:02:33 dist-test-slave-8tdl krb5kdc[24716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172553, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:02:33.391218 24801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:33.379012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:49169 (local address 127.13.207.62:40417)
0808 07:02:33.379141 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:33.379143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:33.379403 (+ 260us) server_negotiation.cc:407] Connection header received
0808 07:02:33.379834 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:33.379836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:33.379848 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:33.379936 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:33.381509 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.382093 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.383551 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.383759 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.383839 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:33.387425 (+ 3586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:33.387428 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:33.387428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:33.387438 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:33.389572 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:33.390085 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:33.390086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:33.390087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:33.390153 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:33.390454 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:33.390455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:33.390456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:33.390595 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:33.390674 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:33.391015 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 07:02:33.391112 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260808 07:02:33.391453 25097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:33.378989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:40417 (local address 127.13.207.2:49169)
0808 07:02:33.379159 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:33.379202 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:33.379723 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:33.379994 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:33.379998 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:33.380833 (+ 835us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:33.381287 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:33.381295 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.382199 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.382204 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:33.383424 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:33.383427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.383931 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.383933 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:33.384071 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:33.384808 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:33.384819 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:33.387310 (+ 2491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:33.389691 (+ 2381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:33.389702 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:33.389705 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:33.389993 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:33.390232 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:33.390233 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:33.390233 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:33.390374 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:33.390664 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:33.390671 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:33.390870 (+ 199us) client_negotiation.cc:769] Sending connection context
0808 07:02:33.390995 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 07:02:33.391182 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260808 07:02:33.392081 25095 heartbeater.cc:347] Connected to a master server at 127.13.207.62:40417
I20260808 07:02:33.392222 25095 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:33.392491 25095 heartbeater.cc:511] Master 127.13.207.62:40417 requested a full tablet report, sending...
I20260808 07:02:33.393190 24761 ts_manager.cc:194] Registered new tserver with Master: 2f2e4f772d7a414099f365ab5bcfbdce (127.13.207.2:38825)
I20260808 07:02:33.393690 24761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:49169
WARNING: no policy specified for kudu/127.13.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
I20260808 07:02:33.423240 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40417
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:43107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:02:33.514344 25100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:02:33.514593 25100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:02:33.514621 25100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:02:33.515941 25100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:02:33.516000 25100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:02:33.516031 25100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:02:33.517645 25100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:43107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:02:33.517891 25100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:02:33.518141 25100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:02:33.523903 25105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:33.523923 25107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:02:33.523928 25109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:02:33.524283 25100 server_base.cc:1034] running on GCE node
I20260808 07:02:33.524567 25100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:02:33.524887 25100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:02:33.526155 25100 hybrid_clock.cc:648] HybridClock initialized: now 1786172553526200 us; error 95 us; skew 500 ppm
Aug 08 07:02:33 dist-test-slave-8tdl krb5kdc[24716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172553, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:02:33.529695 25100 init.cc:377] Logged in from keytab as kudu/127.13.207.3@KRBTEST.COM (short username kudu)
I20260808 07:02:33.530360 25100 webserver.cc:460] Webserver started at http://127.13.207.3:38175/ using document root <none> and password file <none>
I20260808 07:02:33.530622 25100 fs_manager.cc:359] Metadata directory not provided
I20260808 07:02:33.530719 25100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:02:33.530902 25100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:02:33.532047 25100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "63d3c449df18410d9840011e39cf4096"
format_stamp: "Formatted at 2026-08-08 07:02:33 on dist-test-slave-8tdl"
server_key: "3280b8c35fdc3d8c82a3e876c86a28f7"
server_key_iv: "8720c91956de553f6bc2b4e2886c1a16"
server_key_version: "encryptionkey@0"
I20260808 07:02:33.532464 25100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "63d3c449df18410d9840011e39cf4096"
format_stamp: "Formatted at 2026-08-08 07:02:33 on dist-test-slave-8tdl"
server_key: "3280b8c35fdc3d8c82a3e876c86a28f7"
server_key_iv: "8720c91956de553f6bc2b4e2886c1a16"
server_key_version: "encryptionkey@0"
I20260808 07:02:33.533951 25100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:02:33.534961 25117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:33.535235 25100 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:02:33.535442 25100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "63d3c449df18410d9840011e39cf4096"
format_stamp: "Formatted at 2026-08-08 07:02:33 on dist-test-slave-8tdl"
server_key: "3280b8c35fdc3d8c82a3e876c86a28f7"
server_key_iv: "8720c91956de553f6bc2b4e2886c1a16"
server_key_version: "encryptionkey@0"
I20260808 07:02:33.535561 25100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:02:33.546448 25100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:02:33.549180 25100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:02:33.549376 25100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:02:33.549692 25100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:02:33.550159 25100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:02:33.550199 25100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:33.550225 25100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:02:33.550276 25100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:02:33.557542 25100 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:33349
I20260808 07:02:33.557624 25233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:33349 every 8 connection(s)
I20260808 07:02:33.558009 25100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:02:33.560835 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 25100
I20260808 07:02:33.560953 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:02:33.561156 14140 external_mini_cluster.cc:1428] Setting key 18aa92e975f617a6a889c25ce24002dd
Aug 08 07:02:33 dist-test-slave-8tdl krb5kdc[24716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172553, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:02:33.569823 24801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:33.559002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:58935 (local address 127.13.207.62:40417)
0808 07:02:33.559084 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:33.559086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:33.559367 (+ 281us) server_negotiation.cc:407] Connection header received
0808 07:02:33.559891 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:33.559892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:33.559905 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:33.559962 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:33.561291 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.562125 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.563316 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.563574 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.563649 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:33.566422 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:33.566424 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:33.566425 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:33.566433 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:33.567960 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:33.568653 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:33.568654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:33.568654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:33.568718 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:33.569082 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:33.569083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:33.569083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:33.569225 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:33.569288 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:33.569659 (+ 371us) server_negotiation.cc:299] Negotiation successful
0808 07:02:33.569748 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 07:02:33.570128 25236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:33.558936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:40417 (local address 127.13.207.3:58935)
0808 07:02:33.559226 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:33.559268 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:33.559784 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:33.560021 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:33.560024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:33.560789 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:02:33.561145 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:33.561153 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.562197 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.562198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:33.563179 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:33.563183 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.563693 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.563694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:33.563821 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:33.564541 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:02:33.564550 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:02:33.566281 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:02:33.568231 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:33.568239 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:33.568242 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:33.568528 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:33.568797 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:02:33.568798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:02:33.568798 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:02:33.568978 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:02:33.569310 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:02:33.569317 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:02:33.569487 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 07:02:33.569602 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 07:02:33.569766 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":174,"threads_started":1}
I20260808 07:02:33.571038 25234 heartbeater.cc:347] Connected to a master server at 127.13.207.62:40417
I20260808 07:02:33.571149 25234 heartbeater.cc:464] Registering TS with master...
I20260808 07:02:33.571379 25234 heartbeater.cc:511] Master 127.13.207.62:40417 requested a full tablet report, sending...
I20260808 07:02:33.572213 24761 ts_manager.cc:194] Registered new tserver with Master: 63d3c449df18410d9840011e39cf4096 (127.13.207.3:33349)
I20260808 07:02:33.572760 24761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:58935
I20260808 07:02:33.573313 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:02:33.582566 24801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:33.575304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35872 (local address 127.13.207.62:40417)
0808 07:02:33.575375 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:33.575377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:33.575388 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:02:33.575438 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:33.575439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:33.575451 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:33.575529 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:02:33.576221 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.576732 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.577496 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.577698 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.577779 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:33.579331 (+ 1552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:02:33.579333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:02:33.579334 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:02:33.579342 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:02:33.581055 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:33.581466 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:33.581467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:33.581468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:33.581560 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:02:33.581919 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:02:33.581920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:02:33.581923 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:02:33.582109 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:02:33.582228 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:33.582361 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 07:02:33.582453 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:02:33.584983 24761 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35872:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:02:33.586872 24761 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:02:33.595590 25245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:33.590376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:33349 (local address 127.0.0.1:57208)
0808 07:02:33.590622 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:33.590668 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:33.590824 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:33.591198 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:33.591202 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:33.591205 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:33.591305 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:33.591311 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.592809 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.592809 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:33.594173 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:33.594176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.595292 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.595293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:33.595368 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:33.595375 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 07:02:33.595427 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:02:33.595483 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":15,"thread_start_us":217,"threads_started":1}
I20260808 07:02:33.595554 25249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:33.590915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43382 (local address 127.13.207.1:36991)
0808 07:02:33.591112 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:33.591117 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:33.591332 (+ 215us) server_negotiation.cc:407] Connection header received
0808 07:02:33.591377 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:33.591379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:33.591395 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:33.591629 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:33.591899 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.593002 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.594271 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.594969 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.595109 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:33.595188 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:33.595378 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 07:02:33.595427 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":85,"threads_started":1}
I20260808 07:02:33.595679 25250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:33.590983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53558 (local address 127.13.207.2:38825)
0808 07:02:33.591165 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:33.591169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:33.591329 (+ 160us) server_negotiation.cc:407] Connection header received
0808 07:02:33.591377 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:33.591379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:33.591395 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:33.591580 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:33.591745 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.592564 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.594291 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.595114 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.595192 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:33.595249 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:33.595535 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 07:02:33.595582 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":74,"threads_started":1}
I20260808 07:02:33.595618 25248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:33.590757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:38825 (local address 127.0.0.1:53558)
0808 07:02:33.591215 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:33.591224 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:33.591268 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:33.591586 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:33.591587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:33.591589 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:33.591643 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:33.591645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.592691 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.592692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:33.594173 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:33.594176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.595290 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.595292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:33.595369 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:33.595373 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:02:33.595427 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 07:02:33.595484 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":106,"thread_start_us":69,"threads_started":1}
I20260808 07:02:33.596405 25247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:33.590640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57208 (local address 127.13.207.3:33349)
0808 07:02:33.590885 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:33.590891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:33.590905 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:02:33.590949 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:33.590951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:33.590969 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:33.591086 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:33.591406 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.592689 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.594292 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.595118 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.595192 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:33.595249 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:33.596255 (+ 1006us) server_negotiation.cc:299] Negotiation successful
0808 07:02:33.596302 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":136,"threads_started":1}
I20260808 07:02:33.595961 25246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:33.590402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:36991 (local address 127.0.0.1:43382)
0808 07:02:33.591100 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:33.591113 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:33.591265 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:33.591733 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:33.591734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:33.591737 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:33.591803 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:33.591805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.593115 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.593116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:33.594148 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:33.594152 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.595111 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.595112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:33.595216 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:33.595232 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 07:02:33.595354 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 07:02:33.595427 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":124,"threads_started":1}
I20260808 07:02:33.597229 25166 tablet_service.cc:1467] Processing CreateTablet for tablet f8ac9f25f2e44e8483a4cb3110797579 (DEFAULT_TABLE table=test-table [id=737ae3aaccf64499810d081946c90a9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:33.597393 25029 tablet_service.cc:1467] Processing CreateTablet for tablet f8ac9f25f2e44e8483a4cb3110797579 (DEFAULT_TABLE table=test-table [id=737ae3aaccf64499810d081946c90a9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:33.597663 25166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8ac9f25f2e44e8483a4cb3110797579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:33.597728 25029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8ac9f25f2e44e8483a4cb3110797579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:33.597657 24889 tablet_service.cc:1467] Processing CreateTablet for tablet f8ac9f25f2e44e8483a4cb3110797579 (DEFAULT_TABLE table=test-table [id=737ae3aaccf64499810d081946c90a9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:02:33.597994 24889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8ac9f25f2e44e8483a4cb3110797579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:02:33.600957 25251 tablet_bootstrap.cc:492] T f8ac9f25f2e44e8483a4cb3110797579 P 30c958bfa5204620b9f9ee6537debe0c: Bootstrap starting.
I20260808 07:02:33.601222 25252 tablet_bootstrap.cc:492] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce: Bootstrap starting.
I20260808 07:02:33.601747 25251 tablet_bootstrap.cc:654] T f8ac9f25f2e44e8483a4cb3110797579 P 30c958bfa5204620b9f9ee6537debe0c: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:33.601836 25252 tablet_bootstrap.cc:654] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:33.601967 25253 tablet_bootstrap.cc:492] T f8ac9f25f2e44e8483a4cb3110797579 P 63d3c449df18410d9840011e39cf4096: Bootstrap starting.
I20260808 07:02:33.602495 25252 log.cc:824] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:33.602495 25251 log.cc:824] T f8ac9f25f2e44e8483a4cb3110797579 P 30c958bfa5204620b9f9ee6537debe0c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:33.602631 25253 tablet_bootstrap.cc:654] T f8ac9f25f2e44e8483a4cb3110797579 P 63d3c449df18410d9840011e39cf4096: Neither blocks nor log segments found. Creating new log.
I20260808 07:02:33.603324 25253 log.cc:824] T f8ac9f25f2e44e8483a4cb3110797579 P 63d3c449df18410d9840011e39cf4096: Log is configured to *not* fsync() on all Append() calls
I20260808 07:02:33.603449 25251 tablet_bootstrap.cc:492] T f8ac9f25f2e44e8483a4cb3110797579 P 30c958bfa5204620b9f9ee6537debe0c: No bootstrap required, opened a new log
I20260808 07:02:33.603449 25252 tablet_bootstrap.cc:492] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce: No bootstrap required, opened a new log
I20260808 07:02:33.603531 25251 ts_tablet_manager.cc:1397] T f8ac9f25f2e44e8483a4cb3110797579 P 30c958bfa5204620b9f9ee6537debe0c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:02:33.603544 25252 ts_tablet_manager.cc:1397] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260808 07:02:33.604203 25253 tablet_bootstrap.cc:492] T f8ac9f25f2e44e8483a4cb3110797579 P 63d3c449df18410d9840011e39cf4096: No bootstrap required, opened a new log
I20260808 07:02:33.604357 25253 ts_tablet_manager.cc:1397] T f8ac9f25f2e44e8483a4cb3110797579 P 63d3c449df18410d9840011e39cf4096: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:02:33.605605 25252 raft_consensus.cc:348] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d3c449df18410d9840011e39cf4096" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33349 } } peers { permanent_uuid: "2f2e4f772d7a414099f365ab5bcfbdce" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38825 } } peers { permanent_uuid: "30c958bfa5204620b9f9ee6537debe0c" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36991 } }
I20260808 07:02:33.605605 25251 raft_consensus.cc:348] T f8ac9f25f2e44e8483a4cb3110797579 P 30c958bfa5204620b9f9ee6537debe0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d3c449df18410d9840011e39cf4096" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33349 } } peers { permanent_uuid: "2f2e4f772d7a414099f365ab5bcfbdce" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38825 } } peers { permanent_uuid: "30c958bfa5204620b9f9ee6537debe0c" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36991 } }
I20260808 07:02:33.605813 25251 raft_consensus.cc:374] T f8ac9f25f2e44e8483a4cb3110797579 P 30c958bfa5204620b9f9ee6537debe0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:33.605813 25252 raft_consensus.cc:374] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:33.605852 25251 raft_consensus.cc:729] T f8ac9f25f2e44e8483a4cb3110797579 P 30c958bfa5204620b9f9ee6537debe0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30c958bfa5204620b9f9ee6537debe0c, State: Initialized, Role: FOLLOWER
I20260808 07:02:33.605852 25252 raft_consensus.cc:729] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f2e4f772d7a414099f365ab5bcfbdce, State: Initialized, Role: FOLLOWER
I20260808 07:02:33.605995 25251 consensus_queue.cc:260] T f8ac9f25f2e44e8483a4cb3110797579 P 30c958bfa5204620b9f9ee6537debe0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d3c449df18410d9840011e39cf4096" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33349 } } peers { permanent_uuid: "2f2e4f772d7a414099f365ab5bcfbdce" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38825 } } peers { permanent_uuid: "30c958bfa5204620b9f9ee6537debe0c" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36991 } }
I20260808 07:02:33.606022 25252 consensus_queue.cc:260] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d3c449df18410d9840011e39cf4096" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33349 } } peers { permanent_uuid: "2f2e4f772d7a414099f365ab5bcfbdce" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38825 } } peers { permanent_uuid: "30c958bfa5204620b9f9ee6537debe0c" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36991 } }
I20260808 07:02:33.606335 25251 ts_tablet_manager.cc:1428] T f8ac9f25f2e44e8483a4cb3110797579 P 30c958bfa5204620b9f9ee6537debe0c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:02:33.606335 25252 ts_tablet_manager.cc:1428] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:02:33.606382 25095 heartbeater.cc:503] Master 127.13.207.62:40417 was elected leader, sending a full tablet report...
I20260808 07:02:33.606513 24956 heartbeater.cc:503] Master 127.13.207.62:40417 was elected leader, sending a full tablet report...
I20260808 07:02:33.606583 25253 raft_consensus.cc:348] T f8ac9f25f2e44e8483a4cb3110797579 P 63d3c449df18410d9840011e39cf4096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d3c449df18410d9840011e39cf4096" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33349 } } peers { permanent_uuid: "2f2e4f772d7a414099f365ab5bcfbdce" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38825 } } peers { permanent_uuid: "30c958bfa5204620b9f9ee6537debe0c" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36991 } }
I20260808 07:02:33.607148 25253 raft_consensus.cc:374] T f8ac9f25f2e44e8483a4cb3110797579 P 63d3c449df18410d9840011e39cf4096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:02:33.607218 25253 raft_consensus.cc:729] T f8ac9f25f2e44e8483a4cb3110797579 P 63d3c449df18410d9840011e39cf4096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63d3c449df18410d9840011e39cf4096, State: Initialized, Role: FOLLOWER
I20260808 07:02:33.607343 25253 consensus_queue.cc:260] T f8ac9f25f2e44e8483a4cb3110797579 P 63d3c449df18410d9840011e39cf4096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d3c449df18410d9840011e39cf4096" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33349 } } peers { permanent_uuid: "2f2e4f772d7a414099f365ab5bcfbdce" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38825 } } peers { permanent_uuid: "30c958bfa5204620b9f9ee6537debe0c" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36991 } }
I20260808 07:02:33.607764 25253 ts_tablet_manager.cc:1428] T f8ac9f25f2e44e8483a4cb3110797579 P 63d3c449df18410d9840011e39cf4096: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260808 07:02:33.607954 25234 heartbeater.cc:503] Master 127.13.207.62:40417 was elected leader, sending a full tablet report...
W20260808 07:02:33.628690 25096 tablet.cc:2369] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:02:33.685652 24957 tablet.cc:2369] T f8ac9f25f2e44e8483a4cb3110797579 P 30c958bfa5204620b9f9ee6537debe0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:02:33.707801 25258 raft_consensus.cc:482] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:02:33.707973 25258 raft_consensus.cc:504] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d3c449df18410d9840011e39cf4096" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33349 } } peers { permanent_uuid: "2f2e4f772d7a414099f365ab5bcfbdce" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38825 } } peers { permanent_uuid: "30c958bfa5204620b9f9ee6537debe0c" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36991 } }
I20260808 07:02:33.708534 25258 leader_election.cc:288] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63d3c449df18410d9840011e39cf4096 (127.13.207.3:33349), 30c958bfa5204620b9f9ee6537debe0c (127.13.207.1:36991)
I20260808 07:02:33.712419 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:33.708800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:33349 (local address 127.13.207.2:33393)
0808 07:02:33.708980 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:33.708996 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:33.709083 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:33.709228 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:33.709228 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:33.709230 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:33.709310 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:33.709312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.710161 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.710162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:33.711110 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:33.711113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.712076 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.712077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:33.712160 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:33.712165 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:02:33.712240 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 07:02:33.712294 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":128,"threads_started":1}
I20260808 07:02:33.712584 25247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:33.708852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:33393 (local address 127.13.207.3:33349)
0808 07:02:33.708904 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:33.708907 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:33.709081 (+ 174us) server_negotiation.cc:407] Connection header received
0808 07:02:33.709138 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:33.709139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:33.709150 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:33.709226 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:33.709385 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.710024 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.711214 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.711889 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.712065 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:33.712079 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:33.712287 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 07:02:33.712496 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:02:33.712739 25097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:33.708779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:36991 (local address 127.13.207.2:57157)
0808 07:02:33.708844 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:33.708858 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:02:33.708923 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:02:33.709172 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:02:33.709173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:02:33.709175 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:02:33.709254 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:33.709256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.710188 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.710190 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:33.711176 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:02:33.711180 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.712445 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.712447 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:02:33.712536 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:33.712540 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:02:33.712602 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 07:02:33.712652 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":13}
I20260808 07:02:33.712930 25249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:33.708957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:57157 (local address 127.13.207.1:36991)
0808 07:02:33.709013 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:33.709015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:33.709025 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:02:33.709058 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:33.709059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:33.709069 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:33.709164 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:02:33.709332 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.710056 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.711280 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.712256 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.712328 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:33.712337 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:33.712805 (+ 468us) server_negotiation.cc:299] Negotiation successful
0808 07:02:33.712861 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 07:02:33.713327 25186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8ac9f25f2e44e8483a4cb3110797579" candidate_uuid: "2f2e4f772d7a414099f365ab5bcfbdce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63d3c449df18410d9840011e39cf4096" is_pre_election: true
I20260808 07:02:33.713543 25186 raft_consensus.cc:2454] T f8ac9f25f2e44e8483a4cb3110797579 P 63d3c449df18410d9840011e39cf4096 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f2e4f772d7a414099f365ab5bcfbdce in term 0.
I20260808 07:02:33.713588 24910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8ac9f25f2e44e8483a4cb3110797579" candidate_uuid: "2f2e4f772d7a414099f365ab5bcfbdce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30c958bfa5204620b9f9ee6537debe0c" is_pre_election: true
I20260808 07:02:33.713796 24910 raft_consensus.cc:2454] T f8ac9f25f2e44e8483a4cb3110797579 P 30c958bfa5204620b9f9ee6537debe0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f2e4f772d7a414099f365ab5bcfbdce in term 0.
I20260808 07:02:33.713856 24979 leader_election.cc:302] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f2e4f772d7a414099f365ab5bcfbdce, 63d3c449df18410d9840011e39cf4096; no voters:
I20260808 07:02:33.714085 25258 raft_consensus.cc:2781] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:02:33.714241 25258 raft_consensus.cc:482] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:02:33.714306 25258 raft_consensus.cc:3037] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:33.715142 25258 raft_consensus.cc:504] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d3c449df18410d9840011e39cf4096" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33349 } } peers { permanent_uuid: "2f2e4f772d7a414099f365ab5bcfbdce" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38825 } } peers { permanent_uuid: "30c958bfa5204620b9f9ee6537debe0c" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36991 } }
I20260808 07:02:33.715332 25258 leader_election.cc:288] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce [CANDIDATE]: Term 1 election: Requested vote from peers 63d3c449df18410d9840011e39cf4096 (127.13.207.3:33349), 30c958bfa5204620b9f9ee6537debe0c (127.13.207.1:36991)
I20260808 07:02:33.715646 24910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8ac9f25f2e44e8483a4cb3110797579" candidate_uuid: "2f2e4f772d7a414099f365ab5bcfbdce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30c958bfa5204620b9f9ee6537debe0c"
I20260808 07:02:33.715672 25186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8ac9f25f2e44e8483a4cb3110797579" candidate_uuid: "2f2e4f772d7a414099f365ab5bcfbdce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63d3c449df18410d9840011e39cf4096"
I20260808 07:02:33.715844 25186 raft_consensus.cc:3037] T f8ac9f25f2e44e8483a4cb3110797579 P 63d3c449df18410d9840011e39cf4096 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:33.715839 24910 raft_consensus.cc:3037] T f8ac9f25f2e44e8483a4cb3110797579 P 30c958bfa5204620b9f9ee6537debe0c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:02:33.716877 25186 raft_consensus.cc:2454] T f8ac9f25f2e44e8483a4cb3110797579 P 63d3c449df18410d9840011e39cf4096 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f2e4f772d7a414099f365ab5bcfbdce in term 1.
I20260808 07:02:33.716919 24910 raft_consensus.cc:2454] T f8ac9f25f2e44e8483a4cb3110797579 P 30c958bfa5204620b9f9ee6537debe0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f2e4f772d7a414099f365ab5bcfbdce in term 1.
I20260808 07:02:33.717173 24979 leader_election.cc:302] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f2e4f772d7a414099f365ab5bcfbdce, 63d3c449df18410d9840011e39cf4096; no voters:
I20260808 07:02:33.717301 25258 raft_consensus.cc:2781] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:02:33.717504 25258 raft_consensus.cc:686] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce [term 1 LEADER]: Becoming Leader. State: Replica: 2f2e4f772d7a414099f365ab5bcfbdce, State: Running, Role: LEADER
I20260808 07:02:33.717624 25258 consensus_queue.cc:237] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d3c449df18410d9840011e39cf4096" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33349 } } peers { permanent_uuid: "2f2e4f772d7a414099f365ab5bcfbdce" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38825 } } peers { permanent_uuid: "30c958bfa5204620b9f9ee6537debe0c" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36991 } }
I20260808 07:02:33.718961 24761 catalog_manager.cc:5495] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f2e4f772d7a414099f365ab5bcfbdce (127.13.207.2). New cstate: current_term: 1 leader_uuid: "2f2e4f772d7a414099f365ab5bcfbdce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d3c449df18410d9840011e39cf4096" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f2e4f772d7a414099f365ab5bcfbdce" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30c958bfa5204620b9f9ee6537debe0c" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36991 } health_report { overall_health: UNKNOWN } } }
I20260808 07:02:33.747416 25250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:33.744630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53564 (local address 127.13.207.2:38825)
0808 07:02:33.744737 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 07:02:33.744739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:02:33.744748 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:02:33.744787 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:02:33.744788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:02:33.744807 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:02:33.744925 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:02:33.745108 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.745592 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.746440 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:02:33.746651 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:02:33.746818 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:02:33.747020 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:02:33.747134 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:02:33.747235 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 07:02:33.747253 (+ 18us) server_negotiation.cc:299] Negotiation successful
0808 07:02:33.747280 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 07:02:33.749907 24910 raft_consensus.cc:1261] T f8ac9f25f2e44e8483a4cb3110797579 P 30c958bfa5204620b9f9ee6537debe0c [term 1 FOLLOWER]: Refusing update from remote peer 2f2e4f772d7a414099f365ab5bcfbdce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:33.749897 25186 raft_consensus.cc:1261] T f8ac9f25f2e44e8483a4cb3110797579 P 63d3c449df18410d9840011e39cf4096 [term 1 FOLLOWER]: Refusing update from remote peer 2f2e4f772d7a414099f365ab5bcfbdce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:02:33.750345 25258 consensus_queue.cc:1035] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce [LEADER]: Connected to new peer: Peer: permanent_uuid: "63d3c449df18410d9840011e39cf4096" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:33.750504 25263 consensus_queue.cc:1035] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce [LEADER]: Connected to new peer: Peer: permanent_uuid: "30c958bfa5204620b9f9ee6537debe0c" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:02:33.753288 25266 mvcc.cc:205] Tried to move back new op lower bound from 7316162780156305408 to 7316162780028116992. Current Snapshot: MvccSnapshot[applied={T|T < 7316162780156305408}]
I20260808 07:02:33.753881 25268 mvcc.cc:205] Tried to move back new op lower bound from 7316162780156305408 to 7316162780028116992. Current Snapshot: MvccSnapshot[applied={T|T < 7316162780156305408}]
I20260808 07:02:33.754086 25267 mvcc.cc:205] Tried to move back new op lower bound from 7316162780156305408 to 7316162780028116992. Current Snapshot: MvccSnapshot[applied={T|T < 7316162780156305408}]
I20260808 07:02:33.757514 24760 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:02:33.757649 24760 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:02:33.758944 24760 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 285ff77506ce4d9ab98c8d72a5713fd4: Sending DeleteTablet for 3 replicas of tablet f8ac9f25f2e44e8483a4cb3110797579
I20260808 07:02:33.759399 24889 tablet_service.cc:1514] Processing DeleteTablet for tablet f8ac9f25f2e44e8483a4cb3110797579 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:33 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:43382
I20260808 07:02:33.759565 25029 tablet_service.cc:1514] Processing DeleteTablet for tablet f8ac9f25f2e44e8483a4cb3110797579 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:33 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:53558
I20260808 07:02:33.759724 25166 tablet_service.cc:1514] Processing DeleteTablet for tablet f8ac9f25f2e44e8483a4cb3110797579 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:02:33 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:57208
I20260808 07:02:33.759810 25276 tablet_replica.cc:330] T f8ac9f25f2e44e8483a4cb3110797579 P 30c958bfa5204620b9f9ee6537debe0c: stopping tablet replica
I20260808 07:02:33.759963 25276 raft_consensus.cc:2229] T f8ac9f25f2e44e8483a4cb3110797579 P 30c958bfa5204620b9f9ee6537debe0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:02:33.759992 25278 tablet_replica.cc:330] T f8ac9f25f2e44e8483a4cb3110797579 P 63d3c449df18410d9840011e39cf4096: stopping tablet replica
I20260808 07:02:33.760123 25276 raft_consensus.cc:2258] T f8ac9f25f2e44e8483a4cb3110797579 P 30c958bfa5204620b9f9ee6537debe0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:33.760195 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 24826
I20260808 07:02:33.761799 25278 raft_consensus.cc:2229] T f8ac9f25f2e44e8483a4cb3110797579 P 63d3c449df18410d9840011e39cf4096 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:02:33.761991 25278 raft_consensus.cc:2258] T f8ac9f25f2e44e8483a4cb3110797579 P 63d3c449df18410d9840011e39cf4096 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:33.762615 25278 ts_tablet_manager.cc:1905] T f8ac9f25f2e44e8483a4cb3110797579 P 63d3c449df18410d9840011e39cf4096: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:33.765722 25278 ts_tablet_manager.cc:1918] T f8ac9f25f2e44e8483a4cb3110797579 P 63d3c449df18410d9840011e39cf4096: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:02:33.765854 25278 log.cc:1196] T f8ac9f25f2e44e8483a4cb3110797579 P 63d3c449df18410d9840011e39cf4096: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/f8ac9f25f2e44e8483a4cb3110797579
I20260808 07:02:33.766213 25278 ts_tablet_manager.cc:1939] T f8ac9f25f2e44e8483a4cb3110797579 P 63d3c449df18410d9840011e39cf4096: Deleting consensus metadata
I20260808 07:02:33.766824 25277 tablet_replica.cc:330] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce: stopping tablet replica
I20260808 07:02:33.766820 24746 catalog_manager.cc:4841] TS 63d3c449df18410d9840011e39cf4096 (127.13.207.3:33349): tablet f8ac9f25f2e44e8483a4cb3110797579 (table test-table [id=737ae3aaccf64499810d081946c90a9a]) successfully deleted
I20260808 07:02:33.766971 25277 raft_consensus.cc:2229] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:02:33.767138 25277 raft_consensus.cc:2258] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:02:33.767616 25277 ts_tablet_manager.cc:1905] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:02:33.771099 25277 ts_tablet_manager.cc:1918] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:02:33.771239 25277 log.cc:1196] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172493058068-14140-0/minicluster-data/ts-1/wal/wals/f8ac9f25f2e44e8483a4cb3110797579
I20260808 07:02:33.771579 25277 ts_tablet_manager.cc:1939] T f8ac9f25f2e44e8483a4cb3110797579 P 2f2e4f772d7a414099f365ab5bcfbdce: Deleting consensus metadata
W20260808 07:02:33.771922 24745 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.207.1:36991 (error 108)
I20260808 07:02:33.772306 24746 catalog_manager.cc:4841] TS 2f2e4f772d7a414099f365ab5bcfbdce (127.13.207.2:38825): tablet f8ac9f25f2e44e8483a4cb3110797579 (table test-table [id=737ae3aaccf64499810d081946c90a9a]) successfully deleted
I20260808 07:02:33.772926 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 24961
I20260808 07:02:33.776057 25248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:02:33.775847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:36991 (local address 127.0.0.1:43390)
0808 07:02:33.775928 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0808 07:02:33.775998 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.1:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 07:02:33.776257 24745 catalog_manager.cc:4584] TS 30c958bfa5204620b9f9ee6537debe0c (127.13.207.1:36991): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f8ac9f25f2e44e8483a4cb3110797579: Network error: Client connection negotiation failed: client connection to 127.13.207.1:36991: connect: Connection refused (error 111)
I20260808 07:02:33.780370 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 25100
I20260808 07:02:33.787847 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 24729
2026-08-08T07:02:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2398 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:02:33 dist-test-slave-8tdl krb5kdc[25282](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:02:33 dist-test-slave-8tdl krb5kdc[25282](info): set up 2 sockets
Aug 08 07:02:33 dist-test-slave-8tdl krb5kdc[25282](info): commencing operation
krb5kdc: starting...
W20260808 07:02:34.946336 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:02:35 dist-test-slave-8tdl krb5kdc[25282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:02:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:02:35Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.207.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.207.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.207.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/rangeradmin_127.13.207.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.207.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/rangeradmin_127.13.207.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.207.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.207.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.207.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/rangerlookup_127.13.207.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.207.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/rangerlookup_127.13.207.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.207.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.207.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.207.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/HTTP_127.13.207.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.207.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/HTTP_127.13.207.20@KRBTEST.COM.keytab.
I20260808 07:02:35.196123 14140 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskzjmV4g/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/postgres -l logfile start
2026-08-08 07:02:38.993 UTC [25312] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 07:02:38.993 UTC [25312] LOG: listening on IPv4 address "127.13.207.20", port 53453
2026-08-08 07:02:39.004 UTC [25312] LOG: listening on Unix socket "/tmp/.s.PGSQL.53453"
2026-08-08 07:02:39.037 UTC [25315] LOG: database system was shut down at 2026-08-08 07:02:36 UTC
2026-08-08 07:02:39.045 UTC [25312] LOG: database system is ready to accept connections
I20260808 07:02:40.061269 14140 mini_postgres.cc:96] Postgres bound to 53453
2026-08-08 07:02:40.069 UTC [25323] FATAL: database "slave" does not exist
127.13.207.20:53453 - accepting connections
I20260808 07:02:40.070417 14140 mini_ranger.cc:161] Starting Ranger...
I20260808 07:02:40.091568 14140 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 07:02:40.923229 14140 mini_ranger.cc:87] Created ranger Postgres database
I20260808 07:02:40.923312 14140 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ranger-admin
2026-08-08 07:02:41,234 [I] DB FLAVOR :POSTGRES
2026-08-08 07:02:41,235 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 07:02:41,235 [I] Checking connection..
2026-08-08 07:02:41,235 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:53453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 07:02:41,531 [I] Checking connection passed.
2026-08-08 07:02:41,531 [I] --------- Verifying version history table ---------
2026-08-08 07:02:41,531 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:53453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:02:41,830 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 07:02:41,830 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:53453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:02:42,145 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 07:02:42,145 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 07:02:42,146 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:53453/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 07:02:42.438 UTC [25389] WARNING: there is no transaction in progress
2026-08-08 07:02:42,447 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 07:02:42,447 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:53453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:02:42,746 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 07:02:42,747 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 07:02:42,747 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:53453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 07:02:43,081 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:53453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 07:02:43,418 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:53453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-8tdl.c.gcp-upstream.internal','N') ;"
2026-08-08 07:02:43,717 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:53453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 07:02:44,046 [I] Table x_portal_user does not exist in database ranger
2026-08-08 07:02:44,046 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:53453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 07:02:44,360 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 07:02:44,361 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:53453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 07:02:44,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:53453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 07:02:44,962 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 07:02:44,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:53453/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 07:02:46.399 UTC [25503] WARNING: there is no transaction in progress
2026-08-08 07:02:46.430 UTC [25503] WARNING: there is no transaction in progress
2026-08-08 07:02:46.456 UTC [25503] WARNING: there is no transaction in progress
2026-08-08 07:02:46.479 UTC [25503] WARNING: there is no transaction in progress
2026-08-08 07:02:46.502 UTC [25503] WARNING: there is no transaction in progress
2026-08-08 07:02:46.528 UTC [25503] WARNING: there is no transaction in progress
2026-08-08 07:02:46.607 UTC [25503] WARNING: there is no transaction in progress
2026-08-08 07:02:46.618 UTC [25503] WARNING: there is no transaction in progress
2026-08-08 07:02:46.640 UTC [25503] WARNING: there is no transaction in progress
2026-08-08 07:02:46.658 UTC [25503] WARNING: there is no transaction in progress
2026-08-08 07:02:46.675 UTC [25503] WARNING: there is no transaction in progress
2026-08-08 07:02:46.694 UTC [25503] WARNING: there is no transaction in progress
2026-08-08 07:02:46.709 UTC [25503] WARNING: there is no transaction in progress
2026-08-08 07:02:46.719 UTC [25503] WARNING: there is no transaction in progress
2026-08-08 07:02:46.730 UTC [25503] WARNING: there is no transaction in progress
2026-08-08 07:02:46.741 UTC [25503] WARNING: there is no transaction in progress
2026-08-08 07:02:47,823 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 07:02:47,823 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:53453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 07:02:48,107 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 07:02:48,107 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:53453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 07:02:48,417 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 07:02:48,417 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:53453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 07:02:48,687 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 07:02:48,687 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:53453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 07:02:48,983 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:53453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 07:02:49,287 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:53453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-8tdl.c.gcp-upstream.internal';"
2026-08-08 07:02:49,569 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 07:02:49,570 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:53453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 07:02:49,844 [I] DB_PATCHES have already been applied
I20260808 07:02:49.850111 14140 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ranger-admin:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskzjmV4g/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 07:02:49.850237 14140 mini_ranger.cc:193] Using host: 127.13.207.20
I20260808 07:02:49.854483 14140 mini_ranger.cc:227] Ranger admin URL: http://127.13.207.20:51307
Aug 08, 2026 7:02:50 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 7:02:50 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 7:02:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 7:02:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 7:02:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 7:02:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 7:02:50 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 07:02:51.086397 14140 mini_ranger.cc:160] Time spent starting Ranger: real 11.016s user 0.004s sys 0.011s
Aug 08, 2026 7:03:16 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 7:03:16 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 08, 2026 7:03:16 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 7:03:16 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 7:03:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 7:03:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 7:03:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 7:03:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 7:03:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 7:03:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 7:03:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 7:03:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 7:03:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 7:03:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 7:03:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 7:03:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 7:03:16 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 7:03:18 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 07:03:18.821369 14140 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.207.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.207.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.207.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/rangerkms_127.13.207.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.207.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/rangerkms_127.13.207.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.207.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.207.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.207.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/HTTP_127.13.207.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.207.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/HTTP_127.13.207.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 07:03:18 dist-test-slave-8tdl krb5kdc[25282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172598, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 07:03:18.945546 14140 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 07:03:18.965543 14140 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 07:03:20.476750 14140 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 07:03:20.476855 14140 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ranger-kms
I20260808 07:03:20.476878 14140 mini_ranger_kms.cc:227] Using postgres at 127.13.207.20:53453
2026-08-08 07:03:21,129 [I] DB FLAVOR :POSTGRES
2026-08-08 07:03:21,129 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 07:03:21,129 [I] Checking connection
2026-08-08 07:03:21,129 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:53453/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 07:03:21,535 [I] connection success
2026-08-08 07:03:21,535 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 07:03:21,535 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 07:03:21,535 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:53453/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 07:03:21,843 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 07:03:21,843 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 07:03:21,843 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 07:03:21,843 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:53453/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 07:03:22,204 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 07:03:22.857861 14140 mini_ranger_kms.cc:310] Created kms service
I20260808 07:03:23.134027 14140 mini_ranger_kms.cc:326] Created kudu user
I20260808 07:03:23.203923 14140 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 07:03:23 dist-test-slave-8tdl krb5kdc[25282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172598, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.207.20@KRBTEST.COM
I20260808 07:03:23.741768 14140 mini_ranger_kms.cc:384] Added ranger policy
I20260808 07:03:23.741930 14140 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ranger-kms:/tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 07:03:23.741957 14140 mini_ranger_kms.cc:240] Using host: 127.13.207.20
I20260808 07:03:23.747977 14140 mini_ranger_kms.cc:276] Ranger KMS PID: 25690
I20260808 07:03:23.748241 14140 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.207.20:59047
Aug 08, 2026 7:03:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 7:03:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 7:03:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 7:03:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 7:03:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 7:03:26 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 07:03:26.411429 14140 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 7.466s user 0.000s sys 0.623s
I20260808 07:03:26.411548 14140 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 07:03:26.411635 14140 mini_ranger_kms.cc:401] 127.13.207.20:59047/kms/v1/keys
Aug 08 07:03:27 dist-test-slave-8tdl krb5kdc[25282](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786172607, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.207.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 07:03:27 dist-test-slave-8tdl krb5kdc[25282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172607, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.207.20@KRBTEST.COM for HTTP/127.13.207.20@KRBTEST.COM
Aug 08 07:03:28 dist-test-slave-8tdl krb5kdc[25282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172607, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.207.20@KRBTEST.COM for HTTP/127.13.207.20@KRBTEST.COM
Aug 08 07:03:31 dist-test-slave-8tdl krb5kdc[25282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:03:31.561704 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:46233
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:46033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:46233
--ranger_config_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.207.20:59047/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:03:31.718459 25742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:03:31.718855 25742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:03:31.718892 25742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:03:31.721071 25742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:03:31.721215 25742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:03:31.721261 25742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:03:31.721293 25742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:03:31.721336 25742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:03:31.725013 25742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:46033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.207.20:59047/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:46233
--ranger_config_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:46233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:03:31.725409 25742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:03:31.725759 25742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:03:31.734652 25750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:31.734572 25748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:31.739235 25747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:31.740149 25742 server_base.cc:1034] running on GCE node
I20260808 07:03:31.740650 25742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:03:31.741294 25742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:03:31.742533 25742 hybrid_clock.cc:648] HybridClock initialized: now 1786172611742514 us; error 44 us; skew 500 ppm
Aug 08 07:03:31 dist-test-slave-8tdl krb5kdc[25282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172611, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:03:31.752103 25742 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:03:31.752735 25742 webserver.cc:460] Webserver started at http://127.13.207.62:34203/ using document root <none> and password file <none>
I20260808 07:03:31.752993 25742 fs_manager.cc:359] Metadata directory not provided
I20260808 07:03:31.753068 25742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:03:31.753244 25742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:03:31 dist-test-slave-8tdl krb5kdc[25282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172611, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for HTTP/127.13.207.20@KRBTEST.COM
I20260808 07:03:31.828437 25742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "098c64c67a9e461f91ae08a051bc1329"
format_stamp: "Formatted at 2026-08-08 07:03:31 on dist-test-slave-8tdl"
server_key: "84621de1c225ba1345b7ffe3c456c0fa"
server_key_iv: "bc3690f5a282d17c069b500da711d654"
server_key_version: "kuduclusterkey@0"
I20260808 07:03:31.829172 25742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "098c64c67a9e461f91ae08a051bc1329"
format_stamp: "Formatted at 2026-08-08 07:03:31 on dist-test-slave-8tdl"
server_key: "84621de1c225ba1345b7ffe3c456c0fa"
server_key_iv: "bc3690f5a282d17c069b500da711d654"
server_key_version: "kuduclusterkey@0"
I20260808 07:03:31.831477 25742 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 07:03:31.851472 25759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:31.851750 25742 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:03:31.851886 25742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "098c64c67a9e461f91ae08a051bc1329"
format_stamp: "Formatted at 2026-08-08 07:03:31 on dist-test-slave-8tdl"
server_key: "84621de1c225ba1345b7ffe3c456c0fa"
server_key_iv: "bc3690f5a282d17c069b500da711d654"
server_key_version: "kuduclusterkey@0"
I20260808 07:03:31.852016 25742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:03:31.874068 25742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:03:31.877398 25742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:03:31.877661 25742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:03:31.895955 25742 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:46233
I20260808 07:03:31.896021 25813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:46233 every 8 connection(s)
I20260808 07:03:31.896574 25742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:03:31.897845 25814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:03:31.901573 25814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329: Bootstrap starting.
I20260808 07:03:31.902938 25814 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329: Neither blocks nor log segments found. Creating new log.
I20260808 07:03:31.903806 25814 log.cc:824] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329: Log is configured to *not* fsync() on all Append() calls
I20260808 07:03:31.904496 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 25742
I20260808 07:03:31.904613 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:03:31.904830 14140 external_mini_cluster.cc:1428] Setting key ae4837cbe80f90396f9dd5c9ee7cead0
I20260808 07:03:31.904979 25814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329: No bootstrap required, opened a new log
I20260808 07:03:31.910483 25814 raft_consensus.cc:348] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "098c64c67a9e461f91ae08a051bc1329" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 46233 } }
I20260808 07:03:31.910778 25814 raft_consensus.cc:374] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:03:31.910842 25814 raft_consensus.cc:729] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098c64c67a9e461f91ae08a051bc1329, State: Initialized, Role: FOLLOWER
I20260808 07:03:31.911011 25814 consensus_queue.cc:260] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "098c64c67a9e461f91ae08a051bc1329" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 46233 } }
I20260808 07:03:31.911144 25814 raft_consensus.cc:388] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:03:31.911193 25814 raft_consensus.cc:482] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:03:31.911243 25814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:03:31.913100 25814 raft_consensus.cc:504] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "098c64c67a9e461f91ae08a051bc1329" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 46233 } }
I20260808 07:03:31.913499 25814 leader_election.cc:302] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 098c64c67a9e461f91ae08a051bc1329; no voters:
I20260808 07:03:31.914186 25814 leader_election.cc:288] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:03:31.914233 25819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:03:31.914541 25819 raft_consensus.cc:686] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329 [term 1 LEADER]: Becoming Leader. State: Replica: 098c64c67a9e461f91ae08a051bc1329, State: Running, Role: LEADER
I20260808 07:03:31.914758 25819 consensus_queue.cc:237] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "098c64c67a9e461f91ae08a051bc1329" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 46233 } }
I20260808 07:03:31.914806 25814 sys_catalog.cc:564] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:03:31.915741 25821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 098c64c67a9e461f91ae08a051bc1329. Latest consensus state: current_term: 1 leader_uuid: "098c64c67a9e461f91ae08a051bc1329" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "098c64c67a9e461f91ae08a051bc1329" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 46233 } } }
I20260808 07:03:31.915858 25821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329 [sys.catalog]: This master's current role is: LEADER
I20260808 07:03:31.916904 25820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "098c64c67a9e461f91ae08a051bc1329" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "098c64c67a9e461f91ae08a051bc1329" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 46233 } } }
I20260808 07:03:31.917003 25820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329 [sys.catalog]: This master's current role is: LEADER
I20260808 07:03:31.918289 25827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 08 07:03:31 dist-test-slave-8tdl krb5kdc[25282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:03:31.924885 25818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:31.905554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40282 (local address 127.13.207.62:46233)
0808 07:03:31.905887 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:31.905899 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:31.906196 (+ 297us) server_negotiation.cc:407] Connection header received
0808 07:03:31.907641 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:31.907653 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:31.907738 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:31.908226 (+ 488us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:03:31.909527 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:31.911257 (+ 1730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:31.913080 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:31.913812 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:31.913960 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:31.919592 (+ 5632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:03:31.919597 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:03:31.919597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:03:31.919612 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:03:31.922758 (+ 3146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:31.923293 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:31.923297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:31.923297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:31.923440 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:31.923719 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:31.923720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:31.923720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:31.924098 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:03:31.924265 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:31.924470 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 07:03:31.924619 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":159,"threads_started":1}
I20260808 07:03:31.926859 25827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:03:31.928135 25814 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 07:03:31.930882 25827 catalog_manager.cc:1347] Generated new cluster ID: 3132ee1dd583403d854d3930ede4c3f6
I20260808 07:03:31.930989 25827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:03:31.953027 25827 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:03:31.954053 25827 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:03:31.968679 25827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329: Generated new TSK 0
I20260808 07:03:31.968935 25827 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
I20260808 07:03:33.171682 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.207.20:59047/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:46233
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:46033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:03:33.351653 25860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:03:33.351950 25860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:03:33.352010 25860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:03:33.353693 25860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:03:33.353803 25860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:03:33.353958 25860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:03:33.356165 25860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:46033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.207.20:59047/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:46233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:03:33.356493 25860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:03:33.356889 25860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:03:33.364841 25865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:33.365360 25866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:33.365727 25869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:33.366364 25860 server_base.cc:1034] running on GCE node
I20260808 07:03:33.366758 25860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:03:33.367151 25860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:03:33.368359 25860 hybrid_clock.cc:648] HybridClock initialized: now 1786172613368341 us; error 34 us; skew 500 ppm
Aug 08 07:03:33 dist-test-slave-8tdl krb5kdc[25282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:03:33.373106 25860 init.cc:377] Logged in from keytab as kudu/127.13.207.1@KRBTEST.COM (short username kudu)
I20260808 07:03:33.373934 25860 webserver.cc:460] Webserver started at http://127.13.207.1:39457/ using document root <none> and password file <none>
I20260808 07:03:33.374259 25860 fs_manager.cc:359] Metadata directory not provided
I20260808 07:03:33.374516 25860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:03:33.374763 25860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:03:33 dist-test-slave-8tdl krb5kdc[25282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for HTTP/127.13.207.20@KRBTEST.COM
I20260808 07:03:33.392792 25860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "70eed86692734591a93aadb30b44eac7"
format_stamp: "Formatted at 2026-08-08 07:03:33 on dist-test-slave-8tdl"
server_key: "10016efda02b0fa08d130426dab2924f"
server_key_iv: "47564a52b2b6e67ffc1890b5f5a8ec0e"
server_key_version: "kuduclusterkey@0"
I20260808 07:03:33.394666 25860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "70eed86692734591a93aadb30b44eac7"
format_stamp: "Formatted at 2026-08-08 07:03:33 on dist-test-slave-8tdl"
server_key: "10016efda02b0fa08d130426dab2924f"
server_key_iv: "47564a52b2b6e67ffc1890b5f5a8ec0e"
server_key_version: "kuduclusterkey@0"
I20260808 07:03:33.398844 25860 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 07:03:33.415588 25876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:33.415850 25860 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:03:33.415952 25860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "70eed86692734591a93aadb30b44eac7"
format_stamp: "Formatted at 2026-08-08 07:03:33 on dist-test-slave-8tdl"
server_key: "10016efda02b0fa08d130426dab2924f"
server_key_iv: "47564a52b2b6e67ffc1890b5f5a8ec0e"
server_key_version: "kuduclusterkey@0"
I20260808 07:03:33.416069 25860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:03:33.440529 25860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:03:33.444360 25860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:03:33.444720 25860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:03:33.445181 25860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:03:33.445811 25860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:03:33.445878 25860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:33.445917 25860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:03:33.445938 25860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:33.475374 25860 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:44611
I20260808 07:03:33.475616 25993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:44611 every 8 connection(s)
I20260808 07:03:33.476298 25860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:03:33.479095 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 25860
I20260808 07:03:33.479224 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:03:33.479482 14140 external_mini_cluster.cc:1428] Setting key 3a2b44d78a01258aa7392e0cf098b865
Aug 08 07:03:33 dist-test-slave-8tdl krb5kdc[25282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:03:33.493494 25997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:33.477672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:55635 (local address 127.13.207.62:46233)
0808 07:03:33.478212 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:33.478215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:33.478817 (+ 602us) server_negotiation.cc:407] Connection header received
0808 07:03:33.480754 (+ 1937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:33.480755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:33.480771 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:33.481071 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:03:33.482048 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:33.482578 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:33.484274 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:33.484540 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:33.484635 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:33.488948 (+ 4313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:03:33.488950 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:03:33.488950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:03:33.488961 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:03:33.491354 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:33.492058 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:33.492060 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:33.492061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:33.492136 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:33.492538 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:33.492539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:33.492540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:33.492725 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:03:33.492788 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:33.493234 (+ 446us) server_negotiation.cc:299] Negotiation successful
0808 07:03:33.493336 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":343,"threads_started":1}
I20260808 07:03:33.494069 25998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:33.477642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:46233 (local address 127.13.207.1:55635)
0808 07:03:33.478295 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:33.478356 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:33.480624 (+ 2268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:33.481167 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:33.481171 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:33.481568 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:03:33.481931 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:33.481936 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:33.482716 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:33.482717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:33.483969 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:33.483973 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:33.484645 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:33.484646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:33.484896 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:33.485809 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:03:33.485822 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:03:33.488774 (+ 2952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:03:33.491506 (+ 2732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:33.491521 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:33.491525 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:33.491944 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:33.492219 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:33.492220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:33.492220 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:33.492427 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:33.492828 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:03:33.492836 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:03:33.493048 (+ 212us) client_negotiation.cc:769] Sending connection context
0808 07:03:33.493603 (+ 555us) client_negotiation.cc:240] Negotiation successful
0808 07:03:33.493793 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":378,"threads_started":1}
I20260808 07:03:33.495610 25994 heartbeater.cc:347] Connected to a master server at 127.13.207.62:46233
I20260808 07:03:33.495730 25994 heartbeater.cc:464] Registering TS with master...
I20260808 07:03:33.495950 25994 heartbeater.cc:511] Master 127.13.207.62:46233 requested a full tablet report, sending...
I20260808 07:03:33.496555 25777 ts_manager.cc:194] Registered new tserver with Master: 70eed86692734591a93aadb30b44eac7 (127.13.207.1:44611)
I20260808 07:03:33.497246 25777 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:55635
WARNING: no policy specified for kudu/127.13.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
I20260808 07:03:33.535657 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.207.20:59047/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:46233
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:46033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:03:33.709180 26001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:03:33.710601 26001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:03:33.711117 26001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:03:33.713327 26001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:03:33.713575 26001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:03:33.713657 26001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:03:33.719275 26001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:46033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.207.20:59047/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:46233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:03:33.719770 26001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:03:33.720184 26001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:03:33.736138 26012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:33.736413 26001 server_base.cc:1034] running on GCE node
W20260808 07:03:33.742257 26008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:33.742892 26010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:33.743393 26001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:03:33.743767 26001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:03:33.744951 26001 hybrid_clock.cc:648] HybridClock initialized: now 1786172613744884 us; error 86 us; skew 500 ppm
Aug 08 07:03:33 dist-test-slave-8tdl krb5kdc[25282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:03:33.751236 26001 init.cc:377] Logged in from keytab as kudu/127.13.207.2@KRBTEST.COM (short username kudu)
I20260808 07:03:33.751992 26001 webserver.cc:460] Webserver started at http://127.13.207.2:41557/ using document root <none> and password file <none>
I20260808 07:03:33.752303 26001 fs_manager.cc:359] Metadata directory not provided
I20260808 07:03:33.752388 26001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:03:33.752540 26001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:03:33 dist-test-slave-8tdl krb5kdc[25282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for HTTP/127.13.207.20@KRBTEST.COM
Aug 08 07:03:33 dist-test-slave-8tdl krb5kdc[25282](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786172613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:03:33.771412 26001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "3c56f77dc47b42939b8f608fd328b57e"
format_stamp: "Formatted at 2026-08-08 07:03:33 on dist-test-slave-8tdl"
server_key: "760ee8094ce2df1e253c1877182043cc"
server_key_iv: "eb8a80c1dfe3e9296d82394c3341fbdb"
server_key_version: "kuduclusterkey@0"
I20260808 07:03:33.772045 26001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "3c56f77dc47b42939b8f608fd328b57e"
format_stamp: "Formatted at 2026-08-08 07:03:33 on dist-test-slave-8tdl"
server_key: "760ee8094ce2df1e253c1877182043cc"
server_key_iv: "eb8a80c1dfe3e9296d82394c3341fbdb"
server_key_version: "kuduclusterkey@0"
I20260808 07:03:33.777140 26001 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 07:03:33.817440 26019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:33.817654 26001 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:03:33.817768 26001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "3c56f77dc47b42939b8f608fd328b57e"
format_stamp: "Formatted at 2026-08-08 07:03:33 on dist-test-slave-8tdl"
server_key: "760ee8094ce2df1e253c1877182043cc"
server_key_iv: "eb8a80c1dfe3e9296d82394c3341fbdb"
server_key_version: "kuduclusterkey@0"
I20260808 07:03:33.817896 26001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:03:33.842916 26001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:03:33.846030 26001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:03:33.846423 26001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:03:33.846755 26001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:03:33.847839 26001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:03:33.847929 26001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:33.847975 26001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:03:33.848013 26001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:33.876976 26001 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:41353
I20260808 07:03:33.877247 26138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:41353 every 8 connection(s)
I20260808 07:03:33.877552 26001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:03:33.881313 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 26001
I20260808 07:03:33.881476 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:03:33.881680 14140 external_mini_cluster.cc:1428] Setting key 5c24c22366c8f5340f16325d320a69e6
Aug 08 07:03:33 dist-test-slave-8tdl krb5kdc[25282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:03:33.900979 25997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:33.879513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:51385 (local address 127.13.207.62:46233)
0808 07:03:33.879572 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:33.879575 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:33.880018 (+ 443us) server_negotiation.cc:407] Connection header received
0808 07:03:33.880893 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:33.880895 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:33.880908 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:33.880974 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:03:33.882054 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:33.882929 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:33.891654 (+ 8725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:33.892048 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:33.892148 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:33.896935 (+ 4787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:03:33.896937 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:03:33.896938 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:03:33.896950 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:03:33.898973 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:33.899678 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:33.899680 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:33.899680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:33.899766 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:33.900101 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:33.900102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:33.900102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:33.900266 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:03:33.900311 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:33.900750 (+ 439us) server_negotiation.cc:299] Negotiation successful
0808 07:03:33.900868 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:03:33.901961 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:33.879061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:46233 (local address 127.13.207.2:51385)
0808 07:03:33.879522 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:33.879641 (+ 119us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:33.880710 (+ 1069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:33.881113 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:33.881118 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:33.881492 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:03:33.881862 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:33.881871 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:33.889907 (+ 8036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:33.889909 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:33.891487 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:33.891494 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:33.892248 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:33.892249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:33.892910 (+ 661us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:33.893880 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:03:33.893894 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:03:33.896681 (+ 2787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:03:33.899112 (+ 2431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:33.899128 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:33.899131 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:33.899565 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:33.899848 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:33.899849 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:33.899850 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:33.900010 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:33.900360 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:03:33.900368 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:03:33.900579 (+ 211us) client_negotiation.cc:769] Sending connection context
0808 07:03:33.901447 (+ 868us) client_negotiation.cc:240] Negotiation successful
0808 07:03:33.901663 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":304,"threads_started":1}
I20260808 07:03:33.903029 26139 heartbeater.cc:347] Connected to a master server at 127.13.207.62:46233
I20260808 07:03:33.903146 26139 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.13.207.3@KRBTEST.COM; defaulting to no policy
I20260808 07:03:33.903362 26139 heartbeater.cc:511] Master 127.13.207.62:46233 requested a full tablet report, sending...
I20260808 07:03:33.903968 25777 ts_manager.cc:194] Registered new tserver with Master: 3c56f77dc47b42939b8f608fd328b57e (127.13.207.2:41353)
I20260808 07:03:33.904511 25777 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:51385
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
I20260808 07:03:33.941646 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.207.20:59047/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:46233
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:46033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:03:34.103914 26146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:03:34.104241 26146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:03:34.104296 26146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:03:34.106235 26146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:03:34.106348 26146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:03:34.106406 26146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:03:34.108834 26146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:46033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.207.20:59047/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:46233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:03:34.109385 26146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:03:34.109963 26146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:03:34.120724 26154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:03:34.120602 26151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:34.121285 26146 server_base.cc:1034] running on GCE node
W20260808 07:03:34.122222 26152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:03:34.122627 26146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:03:34.123030 26146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:03:34.130581 26146 hybrid_clock.cc:648] HybridClock initialized: now 1786172614130535 us; error 57 us; skew 500 ppm
Aug 08 07:03:34 dist-test-slave-8tdl krb5kdc[25282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172614, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:03:34.138252 26146 init.cc:377] Logged in from keytab as kudu/127.13.207.3@KRBTEST.COM (short username kudu)
I20260808 07:03:34.138999 26146 webserver.cc:460] Webserver started at http://127.13.207.3:36837/ using document root <none> and password file <none>
I20260808 07:03:34.139344 26146 fs_manager.cc:359] Metadata directory not provided
I20260808 07:03:34.139498 26146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:03:34.139732 26146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:03:34 dist-test-slave-8tdl krb5kdc[25282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172614, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for HTTP/127.13.207.20@KRBTEST.COM
I20260808 07:03:34.160615 26146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "a2cde3feec5241c69e24b271c7dd49c2"
format_stamp: "Formatted at 2026-08-08 07:03:34 on dist-test-slave-8tdl"
server_key: "e7008954db7aabdeb97a20a7cc48ce9f"
server_key_iv: "5b1ee7895a045b98447ad939fb56ec1a"
server_key_version: "kuduclusterkey@0"
I20260808 07:03:34.161375 26146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "a2cde3feec5241c69e24b271c7dd49c2"
format_stamp: "Formatted at 2026-08-08 07:03:34 on dist-test-slave-8tdl"
server_key: "e7008954db7aabdeb97a20a7cc48ce9f"
server_key_iv: "5b1ee7895a045b98447ad939fb56ec1a"
server_key_version: "kuduclusterkey@0"
I20260808 07:03:34.175403 26146 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.000s sys 0.002s
I20260808 07:03:34.214046 26161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:34.214306 26146 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.001s sys 0.001s
I20260808 07:03:34.214409 26146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "a2cde3feec5241c69e24b271c7dd49c2"
format_stamp: "Formatted at 2026-08-08 07:03:34 on dist-test-slave-8tdl"
server_key: "e7008954db7aabdeb97a20a7cc48ce9f"
server_key_iv: "5b1ee7895a045b98447ad939fb56ec1a"
server_key_version: "kuduclusterkey@0"
I20260808 07:03:34.214548 26146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:03:34.235289 26146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:03:34.238173 26146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:03:34.238374 26146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:03:34.238884 26146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:03:34.239434 26146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:03:34.239492 26146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:34.239527 26146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:03:34.239557 26146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:03:34.263073 26146 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:44249
I20260808 07:03:34.263278 26279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:44249 every 8 connection(s)
I20260808 07:03:34.263754 26146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:03:34.273752 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 26146
I20260808 07:03:34.273896 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:03:34.274111 14140 external_mini_cluster.cc:1428] Setting key cd2aa37ef15081f493500a8de662e4b5
Aug 08 07:03:34 dist-test-slave-8tdl krb5kdc[25282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172614, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:03:34.280478 25997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:34.265073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:40083 (local address 127.13.207.62:46233)
0808 07:03:34.265174 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:34.265177 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:34.265885 (+ 708us) server_negotiation.cc:407] Connection header received
0808 07:03:34.266983 (+ 1098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:34.266986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:34.267001 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:34.267073 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:03:34.268253 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.268804 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.271926 (+ 3122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.272137 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.272215 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:34.276102 (+ 3887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:03:34.276103 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:03:34.276103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:03:34.276112 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:03:34.278557 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:34.279154 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:34.279161 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:34.279162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:34.279238 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:34.279541 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:34.279542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:34.279542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:34.279764 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:03:34.279870 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:34.280268 (+ 398us) server_negotiation.cc:299] Negotiation successful
0808 07:03:34.280379 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 07:03:34.280763 26282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:34.264984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:46233 (local address 127.13.207.3:40083)
0808 07:03:34.265381 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:34.265461 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:34.266844 (+ 1383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:34.267229 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:34.267234 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:34.267687 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:03:34.268091 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:34.268099 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.269350 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.269352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:34.271701 (+ 2349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:34.271714 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.272272 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.272281 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:34.272541 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:34.273447 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:03:34.273459 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:03:34.275963 (+ 2504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:03:34.278721 (+ 2758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:34.278737 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:34.278740 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:34.279050 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:34.279321 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:03:34.279322 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:03:34.279322 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:03:34.279458 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:03:34.279883 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:03:34.279891 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:03:34.280085 (+ 194us) client_negotiation.cc:769] Sending connection context
0808 07:03:34.280228 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 07:03:34.280405 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":241,"threads_started":1}
I20260808 07:03:34.281746 26280 heartbeater.cc:347] Connected to a master server at 127.13.207.62:46233
I20260808 07:03:34.281976 26280 heartbeater.cc:464] Registering TS with master...
I20260808 07:03:34.282382 26280 heartbeater.cc:511] Master 127.13.207.62:46233 requested a full tablet report, sending...
I20260808 07:03:34.283198 25777 ts_manager.cc:194] Registered new tserver with Master: a2cde3feec5241c69e24b271c7dd49c2 (127.13.207.3:44249)
I20260808 07:03:34.284104 25777 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:40083
I20260808 07:03:34.288333 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:03:34.307885 25997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:34.291925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36522 (local address 127.13.207.62:46233)
0808 07:03:34.292038 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:34.292041 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:34.292600 (+ 559us) server_negotiation.cc:407] Connection header received
0808 07:03:34.292814 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:34.292815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:34.292829 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:34.292957 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:03:34.294308 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.294968 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.296609 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.296815 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.296884 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:34.299830 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:03:34.299832 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:03:34.299832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:03:34.299841 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:03:34.301649 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:34.306755 (+ 5106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:34.306758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:34.306759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:34.306883 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:03:34.307197 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:03:34.307198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:03:34.307198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:03:34.307347 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:03:34.307433 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:34.307663 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 07:03:34.307781 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 07:03:34.313547 25777 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36522:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:03:34.316422 25777 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:03:34.328636 26293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:34.320000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:44611 (local address 127.0.0.1:56194)
0808 07:03:34.321160 (+ 1160us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:34.321218 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:34.321692 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:34.322180 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:34.322184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:34.322187 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:03:34.322276 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:34.322278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.325039 (+ 2761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.325041 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:34.326967 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:34.326973 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.328249 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.328251 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:34.328372 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:34.328395 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 07:03:34.328458 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 07:03:34.328534 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1119,"thread_start_us":86,"threads_started":1}
I20260808 07:03:34.331069 26294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:34.320090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56194 (local address 127.13.207.1:44611)
0808 07:03:34.321624 (+ 1534us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:34.321629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:34.321644 (+ 15us) server_negotiation.cc:407] Connection header received
0808 07:03:34.321824 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:34.321828 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:34.321847 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:34.322154 (+ 307us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:03:34.322380 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.324879 (+ 2499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.327116 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.328117 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.330768 (+ 2651us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:34.330872 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:34.330909 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 07:03:34.330967 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1503,"thread_start_us":104,"threads_started":1}
I20260808 07:03:34.332702 25927 tablet_service.cc:1467] Processing CreateTablet for tablet b6c940e1633a4c8c821feac8fa246c53 (DEFAULT_TABLE table=test-table [id=909c058c60ff4137a43e63f976a90813]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:03:34.333227 25927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6c940e1633a4c8c821feac8fa246c53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:03:34.335379 26293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:34.320271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:44249 (local address 127.0.0.1:57852)
0808 07:03:34.329130 (+ 8859us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:34.329142 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:34.329192 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:34.329737 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:34.329738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:34.329740 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:03:34.329783 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:34.329784 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.332137 (+ 2353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.332138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:34.333670 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:34.333673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.335127 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.335129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:34.335201 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:34.335207 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:03:34.335239 (+ 32us) client_negotiation.cc:240] Negotiation successful
0808 07:03:34.335293 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8842,"thread_start_us":83,"threads_started":1}
I20260808 07:03:34.336365 26298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:34.322951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57852 (local address 127.13.207.3:44249)
0808 07:03:34.323147 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:34.323153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:34.329273 (+ 6120us) server_negotiation.cc:407] Connection header received
0808 07:03:34.329334 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:34.329337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:34.329367 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:34.329620 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:03:34.329870 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.331712 (+ 1842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.333809 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.335005 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.335844 (+ 839us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:34.336130 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:34.336167 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 07:03:34.336218 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":106,"threads_started":1}
I20260808 07:03:34.337560 26296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:34.320432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:41353 (local address 127.0.0.1:52458)
0808 07:03:34.330669 (+ 10237us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:34.330712 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:34.332543 (+ 1831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:34.333044 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:34.333045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:34.333048 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:03:34.333100 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:34.333101 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.334385 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.334386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:34.335582 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:34.335585 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.337264 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.337265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:34.337341 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:34.337345 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:03:34.337399 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 07:03:34.337466 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10213,"thread_start_us":114,"threads_started":1}
I20260808 07:03:34.337392 26211 tablet_service.cc:1467] Processing CreateTablet for tablet b6c940e1633a4c8c821feac8fa246c53 (DEFAULT_TABLE table=test-table [id=909c058c60ff4137a43e63f976a90813]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:03:34.337855 26211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6c940e1633a4c8c821feac8fa246c53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:03:34.338177 26297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:34.320783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52458 (local address 127.13.207.2:41353)
0808 07:03:34.332702 (+ 11919us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:34.332708 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:34.332720 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:03:34.332770 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:34.332772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:34.332790 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:34.332933 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:03:34.333232 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.334271 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.335715 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.337058 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.337138 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:34.337221 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:34.338037 (+ 816us) server_negotiation.cc:299] Negotiation successful
0808 07:03:34.338091 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11870,"thread_start_us":87,"threads_started":1}
I20260808 07:03:34.338718 26299 tablet_bootstrap.cc:492] T b6c940e1633a4c8c821feac8fa246c53 P 70eed86692734591a93aadb30b44eac7: Bootstrap starting.
I20260808 07:03:34.339036 26071 tablet_service.cc:1467] Processing CreateTablet for tablet b6c940e1633a4c8c821feac8fa246c53 (DEFAULT_TABLE table=test-table [id=909c058c60ff4137a43e63f976a90813]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:03:34.339601 26071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6c940e1633a4c8c821feac8fa246c53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:03:34.342914 26300 tablet_bootstrap.cc:492] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2: Bootstrap starting.
I20260808 07:03:34.344395 26300 tablet_bootstrap.cc:654] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2: Neither blocks nor log segments found. Creating new log.
I20260808 07:03:34.345067 26300 log.cc:824] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2: Log is configured to *not* fsync() on all Append() calls
I20260808 07:03:34.345163 26299 tablet_bootstrap.cc:654] T b6c940e1633a4c8c821feac8fa246c53 P 70eed86692734591a93aadb30b44eac7: Neither blocks nor log segments found. Creating new log.
I20260808 07:03:34.345932 26299 log.cc:824] T b6c940e1633a4c8c821feac8fa246c53 P 70eed86692734591a93aadb30b44eac7: Log is configured to *not* fsync() on all Append() calls
I20260808 07:03:34.346254 26300 tablet_bootstrap.cc:492] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2: No bootstrap required, opened a new log
I20260808 07:03:34.346334 26300 ts_tablet_manager.cc:1397] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:03:34.350858 26300 raft_consensus.cc:348] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2cde3feec5241c69e24b271c7dd49c2" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 44249 } } peers { permanent_uuid: "3c56f77dc47b42939b8f608fd328b57e" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41353 } } peers { permanent_uuid: "70eed86692734591a93aadb30b44eac7" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 44611 } }
I20260808 07:03:34.351349 26300 raft_consensus.cc:374] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:03:34.351454 26300 raft_consensus.cc:729] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2cde3feec5241c69e24b271c7dd49c2, State: Initialized, Role: FOLLOWER
I20260808 07:03:34.351703 26300 consensus_queue.cc:260] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2cde3feec5241c69e24b271c7dd49c2" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 44249 } } peers { permanent_uuid: "3c56f77dc47b42939b8f608fd328b57e" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41353 } } peers { permanent_uuid: "70eed86692734591a93aadb30b44eac7" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 44611 } }
I20260808 07:03:34.352380 26300 ts_tablet_manager.cc:1428] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 07:03:34.352581 26280 heartbeater.cc:503] Master 127.13.207.62:46233 was elected leader, sending a full tablet report...
I20260808 07:03:34.346727 26301 tablet_bootstrap.cc:492] T b6c940e1633a4c8c821feac8fa246c53 P 3c56f77dc47b42939b8f608fd328b57e: Bootstrap starting.
I20260808 07:03:34.355958 26299 tablet_bootstrap.cc:492] T b6c940e1633a4c8c821feac8fa246c53 P 70eed86692734591a93aadb30b44eac7: No bootstrap required, opened a new log
I20260808 07:03:34.356168 26301 tablet_bootstrap.cc:654] T b6c940e1633a4c8c821feac8fa246c53 P 3c56f77dc47b42939b8f608fd328b57e: Neither blocks nor log segments found. Creating new log.
I20260808 07:03:34.356205 26299 ts_tablet_manager.cc:1397] T b6c940e1633a4c8c821feac8fa246c53 P 70eed86692734591a93aadb30b44eac7: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.000s
I20260808 07:03:34.357168 26301 log.cc:824] T b6c940e1633a4c8c821feac8fa246c53 P 3c56f77dc47b42939b8f608fd328b57e: Log is configured to *not* fsync() on all Append() calls
I20260808 07:03:34.358458 26301 tablet_bootstrap.cc:492] T b6c940e1633a4c8c821feac8fa246c53 P 3c56f77dc47b42939b8f608fd328b57e: No bootstrap required, opened a new log
I20260808 07:03:34.358541 26301 ts_tablet_manager.cc:1397] T b6c940e1633a4c8c821feac8fa246c53 P 3c56f77dc47b42939b8f608fd328b57e: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.004s
I20260808 07:03:34.361665 26299 raft_consensus.cc:348] T b6c940e1633a4c8c821feac8fa246c53 P 70eed86692734591a93aadb30b44eac7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2cde3feec5241c69e24b271c7dd49c2" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 44249 } } peers { permanent_uuid: "3c56f77dc47b42939b8f608fd328b57e" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41353 } } peers { permanent_uuid: "70eed86692734591a93aadb30b44eac7" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 44611 } }
I20260808 07:03:34.362023 26299 raft_consensus.cc:374] T b6c940e1633a4c8c821feac8fa246c53 P 70eed86692734591a93aadb30b44eac7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:03:34.362284 26299 raft_consensus.cc:729] T b6c940e1633a4c8c821feac8fa246c53 P 70eed86692734591a93aadb30b44eac7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70eed86692734591a93aadb30b44eac7, State: Initialized, Role: FOLLOWER
I20260808 07:03:34.362624 26299 consensus_queue.cc:260] T b6c940e1633a4c8c821feac8fa246c53 P 70eed86692734591a93aadb30b44eac7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2cde3feec5241c69e24b271c7dd49c2" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 44249 } } peers { permanent_uuid: "3c56f77dc47b42939b8f608fd328b57e" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41353 } } peers { permanent_uuid: "70eed86692734591a93aadb30b44eac7" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 44611 } }
I20260808 07:03:34.363602 26299 ts_tablet_manager.cc:1428] T b6c940e1633a4c8c821feac8fa246c53 P 70eed86692734591a93aadb30b44eac7: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 07:03:34.363888 25994 heartbeater.cc:503] Master 127.13.207.62:46233 was elected leader, sending a full tablet report...
I20260808 07:03:34.364324 26301 raft_consensus.cc:348] T b6c940e1633a4c8c821feac8fa246c53 P 3c56f77dc47b42939b8f608fd328b57e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2cde3feec5241c69e24b271c7dd49c2" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 44249 } } peers { permanent_uuid: "3c56f77dc47b42939b8f608fd328b57e" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41353 } } peers { permanent_uuid: "70eed86692734591a93aadb30b44eac7" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 44611 } }
I20260808 07:03:34.366804 26301 raft_consensus.cc:374] T b6c940e1633a4c8c821feac8fa246c53 P 3c56f77dc47b42939b8f608fd328b57e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:03:34.366957 26301 raft_consensus.cc:729] T b6c940e1633a4c8c821feac8fa246c53 P 3c56f77dc47b42939b8f608fd328b57e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c56f77dc47b42939b8f608fd328b57e, State: Initialized, Role: FOLLOWER
I20260808 07:03:34.367177 26301 consensus_queue.cc:260] T b6c940e1633a4c8c821feac8fa246c53 P 3c56f77dc47b42939b8f608fd328b57e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2cde3feec5241c69e24b271c7dd49c2" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 44249 } } peers { permanent_uuid: "3c56f77dc47b42939b8f608fd328b57e" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41353 } } peers { permanent_uuid: "70eed86692734591a93aadb30b44eac7" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 44611 } }
I20260808 07:03:34.367987 26301 ts_tablet_manager.cc:1428] T b6c940e1633a4c8c821feac8fa246c53 P 3c56f77dc47b42939b8f608fd328b57e: Time spent starting tablet: real 0.009s user 0.003s sys 0.003s
I20260808 07:03:34.368203 26139 heartbeater.cc:503] Master 127.13.207.62:46233 was elected leader, sending a full tablet report...
W20260808 07:03:34.379172 26140 tablet.cc:2369] T b6c940e1633a4c8c821feac8fa246c53 P 3c56f77dc47b42939b8f608fd328b57e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:03:34.405190 26304 raft_consensus.cc:482] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:03:34.405354 26304 raft_consensus.cc:504] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2cde3feec5241c69e24b271c7dd49c2" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 44249 } } peers { permanent_uuid: "3c56f77dc47b42939b8f608fd328b57e" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41353 } } peers { permanent_uuid: "70eed86692734591a93aadb30b44eac7" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 44611 } }
I20260808 07:03:34.405874 26304 leader_election.cc:288] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c56f77dc47b42939b8f608fd328b57e (127.13.207.2:41353), 70eed86692734591a93aadb30b44eac7 (127.13.207.1:44611)
I20260808 07:03:34.411654 26282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:34.406126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:44611 (local address 127.13.207.3:36353)
0808 07:03:34.406179 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:34.406197 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:34.406253 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:34.407158 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:34.407159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:34.407161 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:03:34.407366 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:34.407369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.408565 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.408567 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:34.409808 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:34.409812 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.411222 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.411223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:34.411400 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:34.411405 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:03:34.411471 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 07:03:34.411540 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":4}
I20260808 07:03:34.412743 26294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:34.406775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:36353 (local address 127.13.207.1:44611)
0808 07:03:34.406903 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:34.406906 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:34.406917 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:03:34.406962 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:34.406963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:34.406974 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:34.407050 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:03:34.407561 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.408467 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.410073 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.410939 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.411163 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:34.411178 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:34.412463 (+ 1285us) server_negotiation.cc:299] Negotiation successful
0808 07:03:34.412665 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260808 07:03:34.413118 25948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c940e1633a4c8c821feac8fa246c53" candidate_uuid: "a2cde3feec5241c69e24b271c7dd49c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70eed86692734591a93aadb30b44eac7" is_pre_election: true
I20260808 07:03:34.413326 25948 raft_consensus.cc:2454] T b6c940e1633a4c8c821feac8fa246c53 P 70eed86692734591a93aadb30b44eac7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2cde3feec5241c69e24b271c7dd49c2 in term 0.
I20260808 07:03:34.413972 26163 leader_election.cc:302] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70eed86692734591a93aadb30b44eac7, a2cde3feec5241c69e24b271c7dd49c2; no voters:
I20260808 07:03:34.414148 26304 raft_consensus.cc:2781] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:03:34.414230 26297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:34.406847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:33339 (local address 127.13.207.2:41353)
0808 07:03:34.407118 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:34.407120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:34.409028 (+ 1908us) server_negotiation.cc:407] Connection header received
0808 07:03:34.409146 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:34.409148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:34.409162 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:34.409195 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:03:34.409854 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.410821 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.412896 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.413468 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.413586 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:34.413607 (+ 21us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:34.414099 (+ 492us) server_negotiation.cc:299] Negotiation successful
0808 07:03:34.414159 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246}
I20260808 07:03:34.414259 26304 raft_consensus.cc:482] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:03:34.414292 26304 raft_consensus.cc:3037] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:03:34.414593 26308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:34.406138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:41353 (local address 127.13.207.3:33339)
0808 07:03:34.408721 (+ 2583us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:34.408735 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:34.409066 (+ 331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:34.409265 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:03:34.409266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:03:34.409268 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:03:34.409692 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:34.409705 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.410985 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.410986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:34.412743 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:03:34.412748 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.413832 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.413833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:03:34.413978 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:34.413984 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:03:34.414355 (+ 371us) client_negotiation.cc:240] Negotiation successful
0808 07:03:34.414423 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2520,"mutex_wait_us":31,"thread_start_us":308,"threads_started":1}
I20260808 07:03:34.415661 26093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c940e1633a4c8c821feac8fa246c53" candidate_uuid: "a2cde3feec5241c69e24b271c7dd49c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c56f77dc47b42939b8f608fd328b57e" is_pre_election: true
I20260808 07:03:34.415926 26093 raft_consensus.cc:2454] T b6c940e1633a4c8c821feac8fa246c53 P 3c56f77dc47b42939b8f608fd328b57e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2cde3feec5241c69e24b271c7dd49c2 in term 0.
I20260808 07:03:34.415959 26304 raft_consensus.cc:504] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2cde3feec5241c69e24b271c7dd49c2" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 44249 } } peers { permanent_uuid: "3c56f77dc47b42939b8f608fd328b57e" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41353 } } peers { permanent_uuid: "70eed86692734591a93aadb30b44eac7" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 44611 } }
I20260808 07:03:34.416133 26304 leader_election.cc:288] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2 [CANDIDATE]: Term 1 election: Requested vote from peers 3c56f77dc47b42939b8f608fd328b57e (127.13.207.2:41353), 70eed86692734591a93aadb30b44eac7 (127.13.207.1:44611)
I20260808 07:03:34.416484 25948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c940e1633a4c8c821feac8fa246c53" candidate_uuid: "a2cde3feec5241c69e24b271c7dd49c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70eed86692734591a93aadb30b44eac7"
I20260808 07:03:34.416594 25948 raft_consensus.cc:3037] T b6c940e1633a4c8c821feac8fa246c53 P 70eed86692734591a93aadb30b44eac7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:03:34.416862 26093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c940e1633a4c8c821feac8fa246c53" candidate_uuid: "a2cde3feec5241c69e24b271c7dd49c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c56f77dc47b42939b8f608fd328b57e"
I20260808 07:03:34.416954 26093 raft_consensus.cc:3037] T b6c940e1633a4c8c821feac8fa246c53 P 3c56f77dc47b42939b8f608fd328b57e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:03:34.418423 25948 raft_consensus.cc:2454] T b6c940e1633a4c8c821feac8fa246c53 P 70eed86692734591a93aadb30b44eac7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2cde3feec5241c69e24b271c7dd49c2 in term 1.
I20260808 07:03:34.418546 26093 raft_consensus.cc:2454] T b6c940e1633a4c8c821feac8fa246c53 P 3c56f77dc47b42939b8f608fd328b57e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2cde3feec5241c69e24b271c7dd49c2 in term 1.
I20260808 07:03:34.418783 26162 leader_election.cc:302] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c56f77dc47b42939b8f608fd328b57e, a2cde3feec5241c69e24b271c7dd49c2; no voters:
I20260808 07:03:34.418965 26304 raft_consensus.cc:2781] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:03:34.419368 26304 raft_consensus.cc:686] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2 [term 1 LEADER]: Becoming Leader. State: Replica: a2cde3feec5241c69e24b271c7dd49c2, State: Running, Role: LEADER
I20260808 07:03:34.419497 26304 consensus_queue.cc:237] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2cde3feec5241c69e24b271c7dd49c2" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 44249 } } peers { permanent_uuid: "3c56f77dc47b42939b8f608fd328b57e" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41353 } } peers { permanent_uuid: "70eed86692734591a93aadb30b44eac7" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 44611 } }
I20260808 07:03:34.421034 25777 catalog_manager.cc:5495] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2cde3feec5241c69e24b271c7dd49c2 (127.13.207.3). New cstate: current_term: 1 leader_uuid: "a2cde3feec5241c69e24b271c7dd49c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2cde3feec5241c69e24b271c7dd49c2" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 44249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c56f77dc47b42939b8f608fd328b57e" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70eed86692734591a93aadb30b44eac7" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 44611 } health_report { overall_health: UNKNOWN } } }
I20260808 07:03:34.452193 26298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:34.448068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57862 (local address 127.13.207.3:44249)
0808 07:03:34.448192 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 07:03:34.448195 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:03:34.448204 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:03:34.448274 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:03:34.448275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:03:34.448319 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:03:34.448432 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:03:34.448824 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.449624 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.450998 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:03:34.451320 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:03:34.451407 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:03:34.451672 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:03:34.451877 (+ 205us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:03:34.451984 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 07:03:34.452048 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 07:03:34.452080 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260808 07:03:34.455751 25948 raft_consensus.cc:1261] T b6c940e1633a4c8c821feac8fa246c53 P 70eed86692734591a93aadb30b44eac7 [term 1 FOLLOWER]: Refusing update from remote peer a2cde3feec5241c69e24b271c7dd49c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:03:34.455917 26093 raft_consensus.cc:1261] T b6c940e1633a4c8c821feac8fa246c53 P 3c56f77dc47b42939b8f608fd328b57e [term 1 FOLLOWER]: Refusing update from remote peer a2cde3feec5241c69e24b271c7dd49c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:03:34.456503 26304 consensus_queue.cc:1035] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70eed86692734591a93aadb30b44eac7" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 44611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:03:34.456821 26309 consensus_queue.cc:1035] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c56f77dc47b42939b8f608fd328b57e" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:03:34.463478 26314 mvcc.cc:205] Tried to move back new op lower bound from 7316163028807471104 to 7316163028663136256. Current Snapshot: MvccSnapshot[applied={T|T < 7316163028807471104}]
I20260808 07:03:34.465643 26313 mvcc.cc:205] Tried to move back new op lower bound from 7316163028807471104 to 7316163028663136256. Current Snapshot: MvccSnapshot[applied={T|T < 7316163028807471104}]
I20260808 07:03:34.467566 25777 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:03:34.467772 25777 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:03:34.468766 25777 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 098c64c67a9e461f91ae08a051bc1329: Sending DeleteTablet for 3 replicas of tablet b6c940e1633a4c8c821feac8fa246c53
I20260808 07:03:34.469233 25927 tablet_service.cc:1514] Processing DeleteTablet for tablet b6c940e1633a4c8c821feac8fa246c53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:03:34 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:56194
I20260808 07:03:34.469635 26071 tablet_service.cc:1514] Processing DeleteTablet for tablet b6c940e1633a4c8c821feac8fa246c53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:03:34 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:52458
I20260808 07:03:34.469708 26211 tablet_service.cc:1514] Processing DeleteTablet for tablet b6c940e1633a4c8c821feac8fa246c53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:03:34 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:57852
I20260808 07:03:34.470546 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 25860
I20260808 07:03:34.478814 26323 tablet_replica.cc:330] T b6c940e1633a4c8c821feac8fa246c53 P 3c56f77dc47b42939b8f608fd328b57e: stopping tablet replica
I20260808 07:03:34.479100 26323 raft_consensus.cc:2229] T b6c940e1633a4c8c821feac8fa246c53 P 3c56f77dc47b42939b8f608fd328b57e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:03:34.479358 26323 raft_consensus.cc:2258] T b6c940e1633a4c8c821feac8fa246c53 P 3c56f77dc47b42939b8f608fd328b57e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:03:34.479358 26322 tablet_replica.cc:330] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2: stopping tablet replica
I20260808 07:03:34.479517 26322 raft_consensus.cc:2229] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:03:34.479679 26322 raft_consensus.cc:2258] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:03:34.480509 26323 ts_tablet_manager.cc:1905] T b6c940e1633a4c8c821feac8fa246c53 P 3c56f77dc47b42939b8f608fd328b57e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:03:34.483412 26323 ts_tablet_manager.cc:1918] T b6c940e1633a4c8c821feac8fa246c53 P 3c56f77dc47b42939b8f608fd328b57e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:03:34.483949 26323 log.cc:1196] T b6c940e1633a4c8c821feac8fa246c53 P 3c56f77dc47b42939b8f608fd328b57e: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-1/wal/wals/b6c940e1633a4c8c821feac8fa246c53
I20260808 07:03:34.484676 26323 ts_tablet_manager.cc:1939] T b6c940e1633a4c8c821feac8fa246c53 P 3c56f77dc47b42939b8f608fd328b57e: Deleting consensus metadata
I20260808 07:03:34.485389 25760 catalog_manager.cc:4841] TS 3c56f77dc47b42939b8f608fd328b57e (127.13.207.2:41353): tablet b6c940e1633a4c8c821feac8fa246c53 (table test-table [id=909c058c60ff4137a43e63f976a90813]) successfully deleted
W20260808 07:03:34.492142 25761 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.207.1:44611 (error 108)
I20260808 07:03:34.492210 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 26001
I20260808 07:03:34.493848 26322 ts_tablet_manager.cc:1905] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:03:34.496567 26322 ts_tablet_manager.cc:1918] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:03:34.496694 26322 log.cc:1196] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/b6c940e1633a4c8c821feac8fa246c53
I20260808 07:03:34.497038 26322 ts_tablet_manager.cc:1939] T b6c940e1633a4c8c821feac8fa246c53 P a2cde3feec5241c69e24b271c7dd49c2: Deleting consensus metadata
I20260808 07:03:34.500805 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 26146
I20260808 07:03:34.505816 26296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:03:34.505662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:44611 (local address 127.0.0.1:56208)
0808 07:03:34.505711 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:34.505764 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.1:44611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260808 07:03:34.507328 25761 catalog_manager.cc:4584] TS 70eed86692734591a93aadb30b44eac7 (127.13.207.1:44611): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b6c940e1633a4c8c821feac8fa246c53: Network error: Client connection negotiation failed: client connection to 127.13.207.1:44611: connect: Connection refused (error 111)
W20260808 07:03:34.514303 26296 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:03:34.513995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:44249 (local address 127.0.0.1:57874)
0808 07:03:34.514049 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:03:34.514061 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:03:34.514103 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:03:34.514232 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.3:44249: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 07:03:34.515184 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 25742
I20260808 07:03:34.525995 14140 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 07:03:34.872211 14140 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 07:03:34.872339 14140 mini_ranger.cc:66] Stopping Ranger...
I20260808 07:03:35.228955 14140 mini_ranger.cc:68] Stopped Ranger
2026-08-08 07:03:35.229 UTC [25312] LOG: received smart shutdown request
2026-08-08 07:03:35.235 UTC [25312] LOG: background worker "logical replication launcher" (PID 25321) exited with exit code 1
2026-08-08 07:03:35.235 UTC [25316] LOG: shutting down
2026-08-08 07:03:35.522 UTC [25312] LOG: database system is shut down
2026-08-08T07:03:35Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (61855 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:03:35 dist-test-slave-8tdl krb5kdc[26333](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:03:35 dist-test-slave-8tdl krb5kdc[26333](info): set up 2 sockets
Aug 08 07:03:35 dist-test-slave-8tdl krb5kdc[26333](info): commencing operation
krb5kdc: starting...
W20260808 07:03:36.833971 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:03:36 dist-test-slave-8tdl krb5kdc[26333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:03:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:03:36Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.207.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.207.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.207.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/rangeradmin_127.13.207.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.207.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/rangeradmin_127.13.207.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.207.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.207.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.207.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/rangerlookup_127.13.207.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.207.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/rangerlookup_127.13.207.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.207.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.207.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.207.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/HTTP_127.13.207.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.207.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/HTTP_127.13.207.20@KRBTEST.COM.keytab.
I20260808 07:03:37.078449 14140 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskzjmV4g/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/postgres -l logfile start
2026-08-08 07:03:40.900 UTC [26365] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 07:03:40.900 UTC [26365] LOG: listening on IPv4 address "127.13.207.20", port 48127
2026-08-08 07:03:40.907 UTC [26365] LOG: listening on Unix socket "/tmp/.s.PGSQL.48127"
2026-08-08 07:03:40.943 UTC [26368] LOG: database system was shut down at 2026-08-08 07:03:38 UTC
2026-08-08 07:03:40.950 UTC [26365] LOG: database system is ready to accept connections
I20260808 07:03:42.096741 14140 mini_postgres.cc:96] Postgres bound to 48127
2026-08-08 07:03:42.105 UTC [26376] FATAL: database "slave" does not exist
127.13.207.20:48127 - accepting connections
I20260808 07:03:42.105671 14140 mini_ranger.cc:161] Starting Ranger...
I20260808 07:03:42.125829 14140 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 07:03:42.898140 14140 mini_ranger.cc:87] Created ranger Postgres database
I20260808 07:03:42.898252 14140 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ranger-admin
2026-08-08 07:03:43,218 [I] DB FLAVOR :POSTGRES
2026-08-08 07:03:43,218 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 07:03:43,219 [I] Checking connection..
2026-08-08 07:03:43,219 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:48127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 07:03:43,505 [I] Checking connection passed.
2026-08-08 07:03:43,505 [I] --------- Verifying version history table ---------
2026-08-08 07:03:43,505 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:48127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:03:43,819 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 07:03:43,819 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:48127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:03:44,106 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 07:03:44,106 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 07:03:44,106 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:48127/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 07:03:44.375 UTC [26443] WARNING: there is no transaction in progress
2026-08-08 07:03:44,384 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 07:03:44,385 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:48127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:03:44,658 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 07:03:44,659 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 07:03:44,659 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:48127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 07:03:44,938 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:48127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 07:03:45,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:48127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-8tdl.c.gcp-upstream.internal','N') ;"
2026-08-08 07:03:45,515 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:48127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 07:03:45,788 [I] Table x_portal_user does not exist in database ranger
2026-08-08 07:03:45,789 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:48127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 07:03:46,070 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 07:03:46,071 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:48127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 07:03:46,377 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:48127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 07:03:46,672 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 07:03:46,672 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:48127/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 07:03:48.062 UTC [26556] WARNING: there is no transaction in progress
2026-08-08 07:03:48.086 UTC [26556] WARNING: there is no transaction in progress
2026-08-08 07:03:48.110 UTC [26556] WARNING: there is no transaction in progress
2026-08-08 07:03:48.139 UTC [26556] WARNING: there is no transaction in progress
2026-08-08 07:03:48.164 UTC [26556] WARNING: there is no transaction in progress
2026-08-08 07:03:48.194 UTC [26556] WARNING: there is no transaction in progress
2026-08-08 07:03:48.275 UTC [26556] WARNING: there is no transaction in progress
2026-08-08 07:03:48.285 UTC [26556] WARNING: there is no transaction in progress
2026-08-08 07:03:48.305 UTC [26556] WARNING: there is no transaction in progress
2026-08-08 07:03:48.318 UTC [26556] WARNING: there is no transaction in progress
2026-08-08 07:03:48.331 UTC [26556] WARNING: there is no transaction in progress
2026-08-08 07:03:48.349 UTC [26556] WARNING: there is no transaction in progress
2026-08-08 07:03:48.361 UTC [26556] WARNING: there is no transaction in progress
2026-08-08 07:03:48.373 UTC [26556] WARNING: there is no transaction in progress
2026-08-08 07:03:48.383 UTC [26556] WARNING: there is no transaction in progress
2026-08-08 07:03:48.393 UTC [26556] WARNING: there is no transaction in progress
2026-08-08 07:03:49,583 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 07:03:49,584 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:48127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 07:03:49,853 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 07:03:49,854 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:48127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 07:03:50,213 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 07:03:50,213 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:48127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 07:03:50,546 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 07:03:50,547 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:48127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 07:03:50,852 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:48127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 07:03:51,188 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:48127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-8tdl.c.gcp-upstream.internal';"
2026-08-08 07:03:51,485 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 07:03:51,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:48127/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 07:03:51,778 [I] DB_PATCHES have already been applied
I20260808 07:03:51.784837 14140 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ranger-admin:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskzjmV4g/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 07:03:51.784931 14140 mini_ranger.cc:193] Using host: 127.13.207.20
I20260808 07:03:51.789724 14140 mini_ranger.cc:227] Ranger admin URL: http://127.13.207.20:51595
Aug 08, 2026 7:03:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 7:03:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 7:03:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 7:03:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 7:03:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 7:03:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 7:03:52 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 07:03:53.021658 14140 mini_ranger.cc:160] Time spent starting Ranger: real 10.916s user 0.000s sys 0.014s
Aug 08, 2026 7:04:19 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 7:04:19 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 08, 2026 7:04:19 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 7:04:19 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 7:04:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 7:04:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 7:04:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 7:04:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 7:04:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 7:04:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 7:04:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 7:04:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 7:04:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 7:04:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 7:04:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 7:04:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 7:04:19 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 7:04:20 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 07:04:21.244571 14140 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.207.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.207.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.207.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/rangerkms_127.13.207.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.207.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/rangerkms_127.13.207.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.207.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.207.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.207.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/HTTP_127.13.207.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.207.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/HTTP_127.13.207.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 07:04:21 dist-test-slave-8tdl krb5kdc[26333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172661, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 07:04:21.355593 14140 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 07:04:21.369741 14140 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 07:04:22.647176 14140 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 07:04:22.647274 14140 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ranger-kms
I20260808 07:04:22.647289 14140 mini_ranger_kms.cc:227] Using postgres at 127.13.207.20:48127
2026-08-08 07:04:22,851 [I] DB FLAVOR :POSTGRES
2026-08-08 07:04:22,851 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 07:04:22,851 [I] Checking connection
2026-08-08 07:04:22,851 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:48127/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 07:04:23,129 [I] connection success
2026-08-08 07:04:23,129 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 07:04:23,129 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 07:04:23,129 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:48127/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 07:04:23,403 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 07:04:23,403 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 07:04:23,403 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 07:04:23,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.207.20:48127/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 07:04:23,732 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 07:04:24.369280 14140 mini_ranger_kms.cc:310] Created kms service
I20260808 07:04:24.614962 14140 mini_ranger_kms.cc:326] Created kudu user
I20260808 07:04:24.666569 14140 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 07:04:24 dist-test-slave-8tdl krb5kdc[26333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172661, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.207.20@KRBTEST.COM
I20260808 07:04:25.157778 14140 mini_ranger_kms.cc:384] Added ranger policy
I20260808 07:04:25.158329 14140 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ranger-kms:/tmp/dist-test-taskzjmV4g/build/release/bin/postgresql.jar:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskzjmV4g/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 07:04:25.158375 14140 mini_ranger_kms.cc:240] Using host: 127.13.207.20
I20260808 07:04:25.164690 14140 mini_ranger_kms.cc:276] Ranger KMS PID: 26744
I20260808 07:04:25.164811 14140 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.207.20:43065
Aug 08, 2026 7:04:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 7:04:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 7:04:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 7:04:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 7:04:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 7:04:26 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 07:04:27.658658 14140 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.303s user 0.000s sys 0.188s
I20260808 07:04:27.658792 14140 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 07:04:27.658841 14140 mini_ranger_kms.cc:401] 127.13.207.20:43065/kms/v1/keys
Aug 08 07:04:27 dist-test-slave-8tdl krb5kdc[26333](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786172667, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.207.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 07:04:28 dist-test-slave-8tdl krb5kdc[26333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172667, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.207.20@KRBTEST.COM for HTTP/127.13.207.20@KRBTEST.COM
Aug 08 07:04:28 dist-test-slave-8tdl krb5kdc[26333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172667, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.207.20@KRBTEST.COM for HTTP/127.13.207.20@KRBTEST.COM
Aug 08 07:04:31 dist-test-slave-8tdl krb5kdc[26333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:04:31.384339 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:41729
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:41729
--ranger_config_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.207.20:43065/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:31.477815 26795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:31.478118 26795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:31.478163 26795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:31.479622 26795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:31.479715 26795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:31.479740 26795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:31.479764 26795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:31.479805 26795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:31.481873 26795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.207.20:43065/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:41729
--ranger_config_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:41729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:31.482203 26795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:31.482465 26795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:31.489676 26801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:31.489667 26800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:31.489786 26795 server_base.cc:1034] running on GCE node
W20260808 07:04:31.489962 26803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:31.490303 26795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:31.490754 26795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:31.491957 26795 hybrid_clock.cc:648] HybridClock initialized: now 1786172671491928 us; error 66 us; skew 500 ppm
Aug 08 07:04:31 dist-test-slave-8tdl krb5kdc[26333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172671, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:31.495424 26795 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:04:31.496039 26795 webserver.cc:460] Webserver started at http://127.13.207.62:33895/ using document root <none> and password file <none>
I20260808 07:04:31.496305 26795 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:31.496381 26795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:31.496556 26795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:04:31 dist-test-slave-8tdl krb5kdc[26333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172671, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for HTTP/127.13.207.20@KRBTEST.COM
I20260808 07:04:31.565193 26795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "c94ae9d39cb643dba7f84091a9e70aa5"
format_stamp: "Formatted at 2026-08-08 07:04:31 on dist-test-slave-8tdl"
server_key: "25067176993feacf3e244b5ce1687e28"
server_key_iv: "7d44a0568d43e38ab2799fc345c1c55b"
server_key_version: "kuduclusterkey@0"
I20260808 07:04:31.565704 26795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "c94ae9d39cb643dba7f84091a9e70aa5"
format_stamp: "Formatted at 2026-08-08 07:04:31 on dist-test-slave-8tdl"
server_key: "25067176993feacf3e244b5ce1687e28"
server_key_iv: "7d44a0568d43e38ab2799fc345c1c55b"
server_key_version: "kuduclusterkey@0"
I20260808 07:04:31.567395 26795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 07:04:31.583837 26810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:31.584045 26795 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:04:31.584153 26795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "c94ae9d39cb643dba7f84091a9e70aa5"
format_stamp: "Formatted at 2026-08-08 07:04:31 on dist-test-slave-8tdl"
server_key: "25067176993feacf3e244b5ce1687e28"
server_key_iv: "7d44a0568d43e38ab2799fc345c1c55b"
server_key_version: "kuduclusterkey@0"
I20260808 07:04:31.584280 26795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:31.599604 26795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:31.602061 26795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:31.602286 26795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:31.608111 26795 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:41729
I20260808 07:04:31.608142 26864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:41729 every 8 connection(s)
I20260808 07:04:31.608620 26795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:31.609453 26865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:31.612154 26865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5: Bootstrap starting.
I20260808 07:04:31.612862 26865 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:31.613080 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 26795
I20260808 07:04:31.613190 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:04:31.613287 26865 log.cc:824] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:31.613387 14140 external_mini_cluster.cc:1428] Setting key 0f2c5b5cb315c0e5140e6176cb425402
I20260808 07:04:31.614674 26865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5: No bootstrap required, opened a new log
I20260808 07:04:31.617605 26865 raft_consensus.cc:348] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94ae9d39cb643dba7f84091a9e70aa5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41729 } }
I20260808 07:04:31.617810 26865 raft_consensus.cc:374] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:31.617842 26865 raft_consensus.cc:729] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c94ae9d39cb643dba7f84091a9e70aa5, State: Initialized, Role: FOLLOWER
I20260808 07:04:31.618019 26865 consensus_queue.cc:260] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94ae9d39cb643dba7f84091a9e70aa5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41729 } }
I20260808 07:04:31.618152 26865 raft_consensus.cc:388] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:31.618211 26865 raft_consensus.cc:482] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:31.618275 26865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:31.619417 26865 raft_consensus.cc:504] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94ae9d39cb643dba7f84091a9e70aa5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41729 } }
I20260808 07:04:31.619603 26865 leader_election.cc:302] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c94ae9d39cb643dba7f84091a9e70aa5; no voters:
I20260808 07:04:31.619882 26865 leader_election.cc:288] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:31.619915 26870 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:31.620201 26870 raft_consensus.cc:686] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [term 1 LEADER]: Becoming Leader. State: Replica: c94ae9d39cb643dba7f84091a9e70aa5, State: Running, Role: LEADER
I20260808 07:04:31.620350 26865 sys_catalog.cc:564] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:31.620350 26870 consensus_queue.cc:237] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94ae9d39cb643dba7f84091a9e70aa5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41729 } }
I20260808 07:04:31.620916 26870 sys_catalog.cc:455] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c94ae9d39cb643dba7f84091a9e70aa5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94ae9d39cb643dba7f84091a9e70aa5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41729 } } }
I20260808 07:04:31.621124 26870 sys_catalog.cc:458] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:31.621325 26870 sys_catalog.cc:455] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c94ae9d39cb643dba7f84091a9e70aa5. Latest consensus state: current_term: 1 leader_uuid: "c94ae9d39cb643dba7f84091a9e70aa5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94ae9d39cb643dba7f84091a9e70aa5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41729 } } }
I20260808 07:04:31.621445 26870 sys_catalog.cc:458] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:31.621685 26879 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:31.621987 26865 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 07:04:31.622298 26879 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 07:04:31 dist-test-slave-8tdl krb5kdc[26333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:04:31.630398 26869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:31.614025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48070 (local address 127.13.207.62:41729)
0808 07:04:31.614201 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:31.614212 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:31.614474 (+ 262us) server_negotiation.cc:407] Connection header received
0808 07:04:31.615209 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:31.615219 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:31.615294 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:31.615476 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:31.616451 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:31.617949 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:31.619320 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:31.620207 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:31.624522 (+ 4315us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:31.625091 (+ 569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:31.625097 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:31.625098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:31.625109 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:31.628174 (+ 3065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:31.628658 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:31.628661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:31.628661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:31.628760 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:31.628985 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:31.628985 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:31.628986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:31.629322 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:31.629893 (+ 571us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:31.629978 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 07:04:31.630148 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
I20260808 07:04:31.636631 26879 catalog_manager.cc:1347] Generated new cluster ID: 84ffb588328f499d954f57a1a509c64a
I20260808 07:04:31.636704 26879 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:31.644963 26879 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:31.645830 26879 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:31.654577 26879 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5: Generated new TSK 0
I20260808 07:04:31.654947 26879 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
I20260808 07:04:32.532122 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.207.20:43065/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:41729
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:32.660512 26913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:32.660898 26913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:32.661005 26913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:32.663064 26913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:32.663201 26913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:32.663302 26913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:04:32.665745 26913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.207.20:43065/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:41729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:32.666302 26913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:32.666668 26913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:32.674292 26918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:32.674292 26921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:32.674758 26919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:32.675556 26913 server_base.cc:1034] running on GCE node
I20260808 07:04:32.675863 26913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:32.676170 26913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:32.679518 26913 hybrid_clock.cc:648] HybridClock initialized: now 1786172672679449 us; error 31 us; skew 500 ppm
Aug 08 07:04:32 dist-test-slave-8tdl krb5kdc[26333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172672, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:32.683174 26913 init.cc:377] Logged in from keytab as kudu/127.13.207.1@KRBTEST.COM (short username kudu)
I20260808 07:04:32.683784 26913 webserver.cc:460] Webserver started at http://127.13.207.1:34151/ using document root <none> and password file <none>
I20260808 07:04:32.684069 26913 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:32.684142 26913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:32.684302 26913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:04:32 dist-test-slave-8tdl krb5kdc[26333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172672, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for HTTP/127.13.207.20@KRBTEST.COM
I20260808 07:04:32.699986 26913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "3c71b272049c41aeb6a6bc5f605007e1"
format_stamp: "Formatted at 2026-08-08 07:04:32 on dist-test-slave-8tdl"
server_key: "873febaa97b28f126a257fe4f2781a71"
server_key_iv: "bed3d6694149db52f4831490a80ddede"
server_key_version: "kuduclusterkey@0"
I20260808 07:04:32.700495 26913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "3c71b272049c41aeb6a6bc5f605007e1"
format_stamp: "Formatted at 2026-08-08 07:04:32 on dist-test-slave-8tdl"
server_key: "873febaa97b28f126a257fe4f2781a71"
server_key_iv: "bed3d6694149db52f4831490a80ddede"
server_key_version: "kuduclusterkey@0"
I20260808 07:04:32.702558 26913 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260808 07:04:32.716188 26928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:32.716424 26913 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:04:32.716545 26913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "3c71b272049c41aeb6a6bc5f605007e1"
format_stamp: "Formatted at 2026-08-08 07:04:32 on dist-test-slave-8tdl"
server_key: "873febaa97b28f126a257fe4f2781a71"
server_key_iv: "bed3d6694149db52f4831490a80ddede"
server_key_version: "kuduclusterkey@0"
I20260808 07:04:32.716655 26913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:32.741250 26913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:32.744647 26913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:32.744910 26913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:32.745278 26913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:32.745810 26913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:32.745915 26913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:32.746032 26913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:32.746121 26913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:32.755016 26913 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:42773
I20260808 07:04:32.755712 26913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:04:32.755934 27046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:42773 every 8 connection(s)
I20260808 07:04:32.762503 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 26913
I20260808 07:04:32.762650 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:04:32.762909 14140 external_mini_cluster.cc:1428] Setting key ad15c180bd98a538400f55ced852305b
Aug 08 07:04:32 dist-test-slave-8tdl krb5kdc[26333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172672, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:04:32.776774 27049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:32.757122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:41729 (local address 127.13.207.1:53125)
0808 07:04:32.757362 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:32.757415 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:32.758528 (+ 1113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:32.759271 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:32.759276 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:32.759621 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:32.759931 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:32.759938 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:32.767249 (+ 7311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:32.767251 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:32.768840 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:32.768847 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:32.769590 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:32.769592 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:32.769768 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:32.770481 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:32.770494 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:32.772466 (+ 1972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:32.774868 (+ 2402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:32.774879 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:32.774883 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:32.775246 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:32.775493 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:32.775494 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:32.775494 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:32.775659 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:32.776005 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:32.776013 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:32.776202 (+ 189us) client_negotiation.cc:769] Sending connection context
0808 07:04:32.776311 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 07:04:32.776504 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":122,"threads_started":1}
I20260808 07:04:32.777781 27050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:32.757497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:53125 (local address 127.13.207.62:41729)
0808 07:04:32.758789 (+ 1292us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:32.758791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:32.758800 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:04:32.758828 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:32.758828 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:32.758842 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:32.759153 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:32.763033 (+ 3880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:32.767030 (+ 3997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:32.769098 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:32.769435 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:32.772597 (+ 3162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:32.772668 (+ 71us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:32.772669 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:32.772669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:32.772674 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:32.774725 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:32.775351 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:32.775352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:32.775352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:32.775420 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:32.775749 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:32.775749 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:32.775750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:32.775906 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:32.777500 (+ 1594us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:32.777551 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 07:04:32.777663 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1261,"thread_start_us":71,"threads_started":1}
I20260808 07:04:32.778503 27047 heartbeater.cc:347] Connected to a master server at 127.13.207.62:41729
I20260808 07:04:32.778614 27047 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:32.778852 27047 heartbeater.cc:511] Master 127.13.207.62:41729 requested a full tablet report, sending...
I20260808 07:04:32.779525 26828 ts_manager.cc:194] Registered new tserver with Master: 3c71b272049c41aeb6a6bc5f605007e1 (127.13.207.1:42773)
I20260808 07:04:32.780403 26828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:53125
WARNING: no policy specified for kudu/127.13.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
I20260808 07:04:32.830287 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.207.20:43065/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:41729
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:32.964545 27054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:32.965014 27054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:32.965150 27054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
Aug 08 07:04:32 dist-test-slave-8tdl krb5kdc[26333](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786172672, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 07:04:32.967504 27054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:32.967813 27054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:32.967941 27054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:04:32.971011 27054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.207.20:43065/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:41729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:32.971778 27054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:32.972210 27054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:32.981403 27064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:32.981416 27062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:32.983500 27054 server_base.cc:1034] running on GCE node
W20260808 07:04:32.984104 27061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:32.984463 27054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:32.984822 27054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:32.985992 27054 hybrid_clock.cc:648] HybridClock initialized: now 1786172672985969 us; error 46 us; skew 500 ppm
Aug 08 07:04:32 dist-test-slave-8tdl krb5kdc[26333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172672, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:32.989719 27054 init.cc:377] Logged in from keytab as kudu/127.13.207.2@KRBTEST.COM (short username kudu)
I20260808 07:04:32.990562 27054 webserver.cc:460] Webserver started at http://127.13.207.2:41753/ using document root <none> and password file <none>
I20260808 07:04:32.991024 27054 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:32.991170 27054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:32.991466 27054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:04:32 dist-test-slave-8tdl krb5kdc[26333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172672, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for HTTP/127.13.207.20@KRBTEST.COM
I20260808 07:04:33.012720 27054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "a941acc5aaf4474c82c6c735e0abfa4a"
format_stamp: "Formatted at 2026-08-08 07:04:33 on dist-test-slave-8tdl"
server_key: "ae7a9f509adc2960a17705ace1ae7aee"
server_key_iv: "1c88e812c739c1537f84a4d301840ebc"
server_key_version: "kuduclusterkey@0"
I20260808 07:04:33.013530 27054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "a941acc5aaf4474c82c6c735e0abfa4a"
format_stamp: "Formatted at 2026-08-08 07:04:33 on dist-test-slave-8tdl"
server_key: "ae7a9f509adc2960a17705ace1ae7aee"
server_key_iv: "1c88e812c739c1537f84a4d301840ebc"
server_key_version: "kuduclusterkey@0"
I20260808 07:04:33.017980 27054 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.002s
I20260808 07:04:33.035899 27071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:33.036284 27054 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 07:04:33.036407 27054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "a941acc5aaf4474c82c6c735e0abfa4a"
format_stamp: "Formatted at 2026-08-08 07:04:33 on dist-test-slave-8tdl"
server_key: "ae7a9f509adc2960a17705ace1ae7aee"
server_key_iv: "1c88e812c739c1537f84a4d301840ebc"
server_key_version: "kuduclusterkey@0"
I20260808 07:04:33.036502 27054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:33.053047 27054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:33.055940 27054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:33.056206 27054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:33.056668 27054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:33.057302 27054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:33.057418 27054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:33.057502 27054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:33.057566 27054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:33.068240 27054 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:40445
I20260808 07:04:33.068846 27054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:04:33.069018 27191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:40445 every 8 connection(s)
Aug 08 07:04:33 dist-test-slave-8tdl krb5kdc[26333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172672, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:04:33.079082 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 27054
I20260808 07:04:33.079191 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:04:33.079392 14140 external_mini_cluster.cc:1428] Setting key 8450b57ab0f6034a8b5d2f86cb8450c4
I20260808 07:04:33.081954 27050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:33.069797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:38653 (local address 127.13.207.62:41729)
0808 07:04:33.069851 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:33.069853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:33.070120 (+ 267us) server_negotiation.cc:407] Connection header received
0808 07:04:33.070881 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:33.070883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:33.070902 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:33.071002 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:33.071832 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:33.072621 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:33.074159 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:33.074513 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:33.074567 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:33.077696 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:33.077698 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:33.077698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:33.077709 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:33.080002 (+ 2293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:33.080712 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:33.080714 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:33.080714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:33.080785 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:33.081108 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:33.081109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:33.081110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:33.081253 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:33.081296 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:33.081726 (+ 430us) server_negotiation.cc:299] Negotiation successful
0808 07:04:33.081832 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:04:33.082839 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:33.069765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:41729 (local address 127.13.207.2:38653)
0808 07:04:33.069940 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:33.069994 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:33.070754 (+ 760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:33.071096 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:33.071101 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:33.071394 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:33.071697 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:33.071704 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:33.072759 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:33.072760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:33.074043 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:33.074046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:33.074720 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:33.074721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:33.074910 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:33.075624 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:33.075635 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:33.077548 (+ 1913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:33.080196 (+ 2648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:33.080208 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:33.080212 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:33.080604 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:33.080857 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:33.080858 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:33.080858 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:33.081016 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:33.081346 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:33.081354 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:33.081558 (+ 204us) client_negotiation.cc:769] Sending connection context
0808 07:04:33.082334 (+ 776us) client_negotiation.cc:240] Negotiation successful
0808 07:04:33.082548 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260808 07:04:33.083614 27192 heartbeater.cc:347] Connected to a master server at 127.13.207.62:41729
I20260808 07:04:33.083774 27192 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:33.084055 27192 heartbeater.cc:511] Master 127.13.207.62:41729 requested a full tablet report, sending...
I20260808 07:04:33.084568 26828 ts_manager.cc:194] Registered new tserver with Master: a941acc5aaf4474c82c6c735e0abfa4a (127.13.207.2:40445)
I20260808 07:04:33.085063 26828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:38653
WARNING: no policy specified for kudu/127.13.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
I20260808 07:04:33.127668 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.207.20:43065/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:41729
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:33.262217 27198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:33.262631 27198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:33.262792 27198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:33.264850 27198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:33.265022 27198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:33.265117 27198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:04:33.267889 27198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.207.20:43065/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:41729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:33.268549 27198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:33.268941 27198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:33.276150 27204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:33.277117 27206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:33.277467 27198 server_base.cc:1034] running on GCE node
W20260808 07:04:33.277823 27203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:33.278242 27198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:33.278615 27198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:33.281944 27198 hybrid_clock.cc:648] HybridClock initialized: now 1786172673281914 us; error 44 us; skew 500 ppm
Aug 08 07:04:33 dist-test-slave-8tdl krb5kdc[26333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172673, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:33.285614 27198 init.cc:377] Logged in from keytab as kudu/127.13.207.3@KRBTEST.COM (short username kudu)
I20260808 07:04:33.286496 27198 webserver.cc:460] Webserver started at http://127.13.207.3:35659/ using document root <none> and password file <none>
I20260808 07:04:33.286893 27198 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:33.287027 27198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:33.287240 27198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:04:33 dist-test-slave-8tdl krb5kdc[26333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172673, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for HTTP/127.13.207.20@KRBTEST.COM
I20260808 07:04:33.307312 27198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "29c544fac2f7423483a6055125a347ce"
format_stamp: "Formatted at 2026-08-08 07:04:33 on dist-test-slave-8tdl"
server_key: "dc11190307c1c57fb8d1d5d6cf1dd1eb"
server_key_iv: "045671153b66298f596d04f16b320640"
server_key_version: "kuduclusterkey@0"
I20260808 07:04:33.307861 27198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "29c544fac2f7423483a6055125a347ce"
format_stamp: "Formatted at 2026-08-08 07:04:33 on dist-test-slave-8tdl"
server_key: "dc11190307c1c57fb8d1d5d6cf1dd1eb"
server_key_iv: "045671153b66298f596d04f16b320640"
server_key_version: "kuduclusterkey@0"
I20260808 07:04:33.309585 27198 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:33.325091 27213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:33.325371 27198 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:04:33.325475 27198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "29c544fac2f7423483a6055125a347ce"
format_stamp: "Formatted at 2026-08-08 07:04:33 on dist-test-slave-8tdl"
server_key: "dc11190307c1c57fb8d1d5d6cf1dd1eb"
server_key_iv: "045671153b66298f596d04f16b320640"
server_key_version: "kuduclusterkey@0"
I20260808 07:04:33.325603 27198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:33.337889 27198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:33.340402 27198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:33.340576 27198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:33.340863 27198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:33.341352 27198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:33.341455 27198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:33.341533 27198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:33.341598 27198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:33.351102 27198 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:39831
I20260808 07:04:33.351841 27198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:04:33.362937 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 27198
I20260808 07:04:33.363103 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:04:33.363341 14140 external_mini_cluster.cc:1428] Setting key f63b33292debef5592fbfffce537fbc1
Aug 08 07:04:33 dist-test-slave-8tdl krb5kdc[26333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172673, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:04:33.376673 27050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:33.363273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:37109 (local address 127.13.207.62:41729)
0808 07:04:33.363321 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:33.363323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:33.363971 (+ 648us) server_negotiation.cc:407] Connection header received
0808 07:04:33.364901 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:33.364902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:33.364915 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:33.364945 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:33.366081 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:33.366878 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:33.368516 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:33.368827 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:33.368870 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:33.372471 (+ 3601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:33.372486 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:33.372486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:33.372497 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:33.374790 (+ 2293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:33.375492 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:33.375493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:33.375494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:33.375561 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:33.375873 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:33.375874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:33.375874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:33.376018 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:33.376063 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:33.376462 (+ 399us) server_negotiation.cc:299] Negotiation successful
0808 07:04:33.376562 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:04:33.377660 27339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:33.362833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:41729 (local address 127.13.207.3:37109)
0808 07:04:33.363604 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:33.363655 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:33.364772 (+ 1117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:33.365146 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:33.365151 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:33.365616 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:33.365958 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:33.365965 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:33.367001 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:33.367002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:33.368383 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:33.368387 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:33.368971 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:33.368972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:33.369155 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:33.369898 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:33.369908 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:33.372310 (+ 2402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:33.374925 (+ 2615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:33.374938 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:33.374941 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:33.375377 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:33.375635 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:33.375636 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:33.375636 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:33.375787 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:33.376108 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:33.376116 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:33.376299 (+ 183us) client_negotiation.cc:769] Sending connection context
0808 07:04:33.377090 (+ 791us) client_negotiation.cc:240] Negotiation successful
0808 07:04:33.377291 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":719,"thread_start_us":164,"threads_started":1}
I20260808 07:04:33.378664 27336 heartbeater.cc:347] Connected to a master server at 127.13.207.62:41729
I20260808 07:04:33.378909 27336 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:33.379181 27336 heartbeater.cc:511] Master 127.13.207.62:41729 requested a full tablet report, sending...
I20260808 07:04:33.379792 26828 ts_manager.cc:194] Registered new tserver with Master: 29c544fac2f7423483a6055125a347ce (127.13.207.3:39831)
I20260808 07:04:33.380250 26828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:37109
I20260808 07:04:33.383490 27335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:39831 every 8 connection(s)
I20260808 07:04:33.386333 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:04:33.387164 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 26913
I20260808 07:04:33.409698 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 27054
I20260808 07:04:33.417325 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 27198
I20260808 07:04:33.425580 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 26795
I20260808 07:04:33.437642 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:41729
--webserver_interface=127.13.207.62
--webserver_port=33895
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:41729
--ranger_config_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.207.20:43065/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.207.20:43065/kms with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:33.537063 27340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:33.537343 27340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:33.537386 27340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:33.538825 27340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:33.538935 27340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:33.538968 27340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:33.539006 27340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:33.539055 27340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:33.541289 27340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.207.20:43065/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:41729
--ranger_config_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:41729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=33895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:33.541601 27340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:33.541836 27340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:33.547497 27346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:33.547498 27345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:33.547760 27340 server_base.cc:1034] running on GCE node
W20260808 07:04:33.547555 27348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:33.548096 27340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:33.548463 27340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:33.549646 27340 hybrid_clock.cc:648] HybridClock initialized: now 1786172673549629 us; error 30 us; skew 500 ppm
Aug 08 07:04:33 dist-test-slave-8tdl krb5kdc[26333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172673, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:33.552606 27340 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:04:33.553094 27340 webserver.cc:460] Webserver started at http://127.13.207.62:33895/ using document root <none> and password file <none>
I20260808 07:04:33.553362 27340 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:33.553445 27340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 07:04:33 dist-test-slave-8tdl krb5kdc[26333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172673, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for HTTP/127.13.207.20@KRBTEST.COM
I20260808 07:04:33.650451 27340 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260808 07:04:33.651773 27357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:33.652348 27340 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:04:33.652468 27340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "c94ae9d39cb643dba7f84091a9e70aa5"
format_stamp: "Formatted at 2026-08-08 07:04:31 on dist-test-slave-8tdl"
server_key: "25067176993feacf3e244b5ce1687e28"
server_key_iv: "7d44a0568d43e38ab2799fc345c1c55b"
server_key_version: "kuduclusterkey@0"
I20260808 07:04:33.653019 27340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:33.674399 27340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:33.676760 27340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:33.676908 27340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:33.682125 27340 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:41729
I20260808 07:04:33.682165 27411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:41729 every 8 connection(s)
I20260808 07:04:33.682631 27340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:33.684011 27412 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:04:33.684710 27412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5: Bootstrap starting.
I20260808 07:04:33.690805 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 27340
I20260808 07:04:33.691187 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.207.20:43065/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:42773
--local_ip_for_outbound_sockets=127.13.207.1
--tserver_master_addrs=127.13.207.62:41729
--webserver_port=34151
--webserver_interface=127.13.207.1
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.207.20:43065/kms with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:04:33.697858 27412 log.cc:824] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:33.699807 27412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:04:33.700212 27412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5: Bootstrap complete.
I20260808 07:04:33.703197 27412 raft_consensus.cc:348] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94ae9d39cb643dba7f84091a9e70aa5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41729 } }
I20260808 07:04:33.703517 27412 raft_consensus.cc:729] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c94ae9d39cb643dba7f84091a9e70aa5, State: Initialized, Role: FOLLOWER
I20260808 07:04:33.703737 27412 consensus_queue.cc:260] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94ae9d39cb643dba7f84091a9e70aa5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41729 } }
I20260808 07:04:33.703828 27412 raft_consensus.cc:388] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:33.703886 27412 raft_consensus.cc:482] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:33.703938 27412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:04:33.705142 27412 raft_consensus.cc:504] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94ae9d39cb643dba7f84091a9e70aa5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41729 } }
I20260808 07:04:33.705282 27412 leader_election.cc:302] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c94ae9d39cb643dba7f84091a9e70aa5; no voters:
I20260808 07:04:33.705488 27412 leader_election.cc:288] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 07:04:33.705615 27416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 07:04:33.705925 27416 raft_consensus.cc:686] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [term 2 LEADER]: Becoming Leader. State: Replica: c94ae9d39cb643dba7f84091a9e70aa5, State: Running, Role: LEADER
I20260808 07:04:33.705930 27412 sys_catalog.cc:564] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:33.706136 27416 consensus_queue.cc:237] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94ae9d39cb643dba7f84091a9e70aa5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41729 } }
I20260808 07:04:33.706527 27417 sys_catalog.cc:455] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c94ae9d39cb643dba7f84091a9e70aa5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94ae9d39cb643dba7f84091a9e70aa5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41729 } } }
I20260808 07:04:33.706743 27417 sys_catalog.cc:458] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:33.707336 27423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:33.707408 27418 sys_catalog.cc:455] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c94ae9d39cb643dba7f84091a9e70aa5. Latest consensus state: current_term: 2 leader_uuid: "c94ae9d39cb643dba7f84091a9e70aa5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94ae9d39cb643dba7f84091a9e70aa5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 41729 } } }
I20260808 07:04:33.707540 27418 sys_catalog.cc:458] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:33.707973 27423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:33.708108 27423 catalog_manager.cc:1259] Loaded cluster ID: 84ffb588328f499d954f57a1a509c64a
I20260808 07:04:33.708148 27423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:33.708588 27412 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 07:04:33.709448 27423 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:33.709767 27423 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5: Loaded TSK: 0
I20260808 07:04:33.710029 27423 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:04:33.823537 27414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:33.823904 27414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:33.823951 27414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:33.826366 27414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:33.826488 27414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:33.826538 27414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:04:33.829479 27414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.207.20:43065/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:42773
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=34151
--tserver_master_addrs=127.13.207.62:41729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:33.829970 27414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:33.830287 27414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:33.837982 27454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:33.838454 27451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:33.838485 27452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:33.839486 27414 server_base.cc:1034] running on GCE node
I20260808 07:04:33.839775 27414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:33.840102 27414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:33.841293 27414 hybrid_clock.cc:648] HybridClock initialized: now 1786172673841262 us; error 47 us; skew 500 ppm
Aug 08 07:04:33 dist-test-slave-8tdl krb5kdc[26333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172673, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:33.847038 27414 init.cc:377] Logged in from keytab as kudu/127.13.207.1@KRBTEST.COM (short username kudu)
I20260808 07:04:33.847911 27414 webserver.cc:460] Webserver started at http://127.13.207.1:34151/ using document root <none> and password file <none>
I20260808 07:04:33.848305 27414 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:33.848448 27414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 07:04:33 dist-test-slave-8tdl krb5kdc[26333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172673, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for HTTP/127.13.207.20@KRBTEST.COM
I20260808 07:04:33.881197 27414 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.001s sys 0.000s
I20260808 07:04:33.882632 27462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:33.882929 27414 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:33.883049 27414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "3c71b272049c41aeb6a6bc5f605007e1"
format_stamp: "Formatted at 2026-08-08 07:04:32 on dist-test-slave-8tdl"
server_key: "873febaa97b28f126a257fe4f2781a71"
server_key_iv: "bed3d6694149db52f4831490a80ddede"
server_key_version: "kuduclusterkey@0"
I20260808 07:04:33.883556 27414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:33.905105 27414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:33.908025 27414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:33.908282 27414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:33.908651 27414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:33.909197 27414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:33.909292 27414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:33.909402 27414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:33.909439 27414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:33.918520 27414 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:42773
I20260808 07:04:33.919450 27414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:04:33.921115 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 27414
I20260808 07:04:33.921514 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.207.20:43065/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:40445
--local_ip_for_outbound_sockets=127.13.207.2
--tserver_master_addrs=127.13.207.62:41729
--webserver_port=41753
--webserver_interface=127.13.207.2
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.207.20:43065/kms with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:04:33.922706 27584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:42773 every 8 connection(s)
Aug 08 07:04:33 dist-test-slave-8tdl krb5kdc[26333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172673, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:04:33.963807 27588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:33.937439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:41729 (local address 127.13.207.1:50655)
0808 07:04:33.942891 (+ 5452us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:33.942999 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:33.943923 (+ 924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:33.944960 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:33.944965 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:33.950914 (+ 5949us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:33.951300 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:33.951312 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:33.954134 (+ 2822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:33.954137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:33.955314 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:33.955321 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:33.956542 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:33.956544 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:33.956694 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:33.957387 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:33.957398 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:33.959367 (+ 1969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:33.961781 (+ 2414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:33.961790 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:33.961793 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:33.962110 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:33.962428 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:33.962428 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:33.962429 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:33.962577 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:33.963114 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:33.963121 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:33.963269 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 07:04:33.963376 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 07:04:33.963567 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5352,"thread_start_us":278,"threads_started":1}
I20260808 07:04:33.964658 27589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:33.937510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:50655 (local address 127.13.207.62:41729)
0808 07:04:33.944186 (+ 6676us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:33.944197 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:33.944240 (+ 43us) server_negotiation.cc:407] Connection header received
0808 07:04:33.944425 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:33.944432 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:33.944514 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:33.944757 (+ 243us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:33.951554 (+ 6797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:33.953073 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:33.955464 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:33.956124 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:33.956289 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:33.959548 (+ 3259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:33.959552 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:33.959553 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:33.959568 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:33.961662 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:33.962214 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:33.962217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:33.962218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:33.962346 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:33.962658 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:33.962659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:33.962660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:33.963001 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:33.963082 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:33.964296 (+ 1214us) server_negotiation.cc:299] Negotiation successful
0808 07:04:33.964438 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6602,"thread_start_us":245,"threads_started":1}
I20260808 07:04:33.966331 27585 heartbeater.cc:347] Connected to a master server at 127.13.207.62:41729
I20260808 07:04:33.966437 27585 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:33.966820 27585 heartbeater.cc:511] Master 127.13.207.62:41729 requested a full tablet report, sending...
W20260808 07:04:33.967798 27585 heartbeater.cc:644] Failed to heartbeat to 127.13.207.62:41729 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 07:04:33.967870 27585 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:33.967940 27585 heartbeater.cc:511] Master 127.13.207.62:41729 requested a full tablet report, sending...
I20260808 07:04:33.968393 27585 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:33.968487 27585 heartbeater.cc:511] Master 127.13.207.62:41729 requested a full tablet report, sending...
W20260808 07:04:33.968864 27585 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:04:34.058957 27587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:34.059276 27587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:34.059332 27587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:34.061391 27587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:34.061482 27587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:34.061535 27587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:04:34.064368 27587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.207.20:43065/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:40445
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=41753
--tserver_master_addrs=127.13.207.62:41729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:34.064858 27587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:34.065263 27587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:34.073177 27597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:34.073711 27594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:34.073913 27595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:34.073951 27587 server_base.cc:1034] running on GCE node
I20260808 07:04:34.074191 27587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:34.074498 27587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:34.075673 27587 hybrid_clock.cc:648] HybridClock initialized: now 1786172674075598 us; error 88 us; skew 500 ppm
Aug 08 07:04:34 dist-test-slave-8tdl krb5kdc[26333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172674, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:34.079730 27587 init.cc:377] Logged in from keytab as kudu/127.13.207.2@KRBTEST.COM (short username kudu)
I20260808 07:04:34.080847 27587 webserver.cc:460] Webserver started at http://127.13.207.2:41753/ using document root <none> and password file <none>
I20260808 07:04:34.081260 27587 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:34.081344 27587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 07:04:34 dist-test-slave-8tdl krb5kdc[26333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172674, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for HTTP/127.13.207.20@KRBTEST.COM
I20260808 07:04:34.097429 27587 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260808 07:04:34.098922 27607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:34.099157 27587 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:34.099284 27587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "a941acc5aaf4474c82c6c735e0abfa4a"
format_stamp: "Formatted at 2026-08-08 07:04:33 on dist-test-slave-8tdl"
server_key: "ae7a9f509adc2960a17705ace1ae7aee"
server_key_iv: "1c88e812c739c1537f84a4d301840ebc"
server_key_version: "kuduclusterkey@0"
I20260808 07:04:34.099709 27587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:34.114387 27587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:34.116888 27587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:34.117085 27587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:34.117422 27587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:34.117954 27587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:34.118010 27587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:34.118057 27587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:34.118093 27587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:34.135524 27587 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:40445
I20260808 07:04:34.136198 27587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:04:34.138126 27730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:40445 every 8 connection(s)
I20260808 07:04:34.141959 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 27587
I20260808 07:04:34.142313 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.207.20:43065/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:39831
--local_ip_for_outbound_sockets=127.13.207.3
--tserver_master_addrs=127.13.207.62:41729
--webserver_port=35659
--webserver_interface=127.13.207.3
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.207.20:43065/kms with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:04:34 dist-test-slave-8tdl krb5kdc[26333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172674, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:04:34.150215 27589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:34.137983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:57763 (local address 127.13.207.62:41729)
0808 07:04:34.138044 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:34.138046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:34.138476 (+ 430us) server_negotiation.cc:407] Connection header received
0808 07:04:34.139254 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:34.139256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:34.139274 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:34.139369 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:34.140207 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:34.141004 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:34.142339 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:34.142718 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:34.142790 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:34.146100 (+ 3310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:34.146102 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:34.146103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:34.146113 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:34.148066 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:34.148837 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:34.148838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:34.148839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:34.148918 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:34.149276 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:34.149278 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:34.149278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:34.149461 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:34.149557 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:34.149986 (+ 429us) server_negotiation.cc:299] Negotiation successful
0808 07:04:34.150110 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:04:34.150484 27733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:34.137624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:41729 (local address 127.13.207.2:57763)
0808 07:04:34.138315 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:34.138363 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:34.139103 (+ 740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:34.139482 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:34.139486 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:34.139748 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:34.140070 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:34.140077 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:34.141148 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:34.141150 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:34.142240 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:34.142244 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:34.142826 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:34.142828 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:34.142961 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:34.143964 (+ 1003us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:34.144029 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:34.145955 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:34.148178 (+ 2223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:34.148190 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:34.148194 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:34.148706 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:34.148994 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:34.148995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:34.148996 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:34.149171 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:34.149559 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:34.149567 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:34.149796 (+ 229us) client_negotiation.cc:769] Sending connection context
0808 07:04:34.149936 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 07:04:34.150144 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":103,"threads_started":1}
I20260808 07:04:34.153338 27731 heartbeater.cc:347] Connected to a master server at 127.13.207.62:41729
I20260808 07:04:34.153498 27731 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:34.153750 27731 heartbeater.cc:511] Master 127.13.207.62:41729 requested a full tablet report, sending...
W20260808 07:04:34.155081 27731 heartbeater.cc:644] Failed to heartbeat to 127.13.207.62:41729 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 07:04:34.155192 27731 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:34.155298 27731 heartbeater.cc:511] Master 127.13.207.62:41729 requested a full tablet report, sending...
I20260808 07:04:34.155977 27731 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:34.156071 27731 heartbeater.cc:511] Master 127.13.207.62:41729 requested a full tablet report, sending...
W20260808 07:04:34.156600 27731 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:04:34.263291 27734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:34.263646 27734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:34.263697 27734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:34.265130 27734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:34.265221 27734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:34.265272 27734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:04:34.267264 27734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.207.20:43065/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:39831
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=35659
--tserver_master_addrs=127.13.207.62:41729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:34.267583 27734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:34.267871 27734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:34.275034 27742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:34.275434 27739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:34.276643 27734 server_base.cc:1034] running on GCE node
W20260808 07:04:34.278924 27740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:34.279384 27734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:34.279960 27734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:34.281183 27734 hybrid_clock.cc:648] HybridClock initialized: now 1786172674281153 us; error 48 us; skew 500 ppm
Aug 08 07:04:34 dist-test-slave-8tdl krb5kdc[26333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172674, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:34.285208 27734 init.cc:377] Logged in from keytab as kudu/127.13.207.3@KRBTEST.COM (short username kudu)
I20260808 07:04:34.286093 27734 webserver.cc:460] Webserver started at http://127.13.207.3:35659/ using document root <none> and password file <none>
I20260808 07:04:34.286492 27734 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:34.286706 27734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 07:04:34 dist-test-slave-8tdl krb5kdc[26333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172674, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for HTTP/127.13.207.20@KRBTEST.COM
I20260808 07:04:34.317170 27734 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:34.318612 27751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:34.318889 27734 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:34.318982 27734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "29c544fac2f7423483a6055125a347ce"
format_stamp: "Formatted at 2026-08-08 07:04:33 on dist-test-slave-8tdl"
server_key: "dc11190307c1c57fb8d1d5d6cf1dd1eb"
server_key_iv: "045671153b66298f596d04f16b320640"
server_key_version: "kuduclusterkey@0"
I20260808 07:04:34.319480 27734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:34.355299 27734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:34.357849 27734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:34.358086 27734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:34.358603 27734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:34.359226 27734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:34.359350 27734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:34.359433 27734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:34.359494 27734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:34.368685 27734 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:39831
I20260808 07:04:34.368777 27870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:39831 every 8 connection(s)
I20260808 07:04:34.369166 27734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:04:34.375479 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 27734
Aug 08 07:04:34 dist-test-slave-8tdl krb5kdc[26333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172674, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:04:34.381862 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:34.370023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:41729 (local address 127.13.207.3:60799)
0808 07:04:34.370212 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:34.370263 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:34.370945 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:34.371346 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:34.371351 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:34.371691 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:34.371990 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:34.371998 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:34.373017 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:34.373018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:34.374356 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:34.374360 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:34.374975 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:34.374977 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:34.375135 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:34.376196 (+ 1061us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:34.376207 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:34.377982 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:34.380049 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:34.380061 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:34.380065 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:34.380382 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:34.380616 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:34.380618 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:34.380618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:34.380749 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:34.381034 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:34.381042 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:34.381240 (+ 198us) client_negotiation.cc:769] Sending connection context
0808 07:04:34.381391 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 07:04:34.381588 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":93,"threads_started":1}
I20260808 07:04:34.382444 27589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:34.370089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:60799 (local address 127.13.207.62:41729)
0808 07:04:34.370174 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:34.370177 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:34.370371 (+ 194us) server_negotiation.cc:407] Connection header received
0808 07:04:34.371126 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:34.371128 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:34.371144 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:34.371230 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:34.372141 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:34.372907 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:34.374487 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:34.374845 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:34.374932 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:34.378102 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:34.378104 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:34.378105 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:34.378113 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:34.379944 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:34.380469 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:34.380470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:34.380471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:34.380538 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:34.380816 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:34.380817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:34.380818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:34.380964 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:34.381041 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:34.382234 (+ 1193us) server_negotiation.cc:299] Negotiation successful
0808 07:04:34.382338 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 07:04:34.383423 27871 heartbeater.cc:347] Connected to a master server at 127.13.207.62:41729
I20260808 07:04:34.383594 27871 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:34.383843 27871 heartbeater.cc:511] Master 127.13.207.62:41729 requested a full tablet report, sending...
W20260808 07:04:34.384677 27871 heartbeater.cc:644] Failed to heartbeat to 127.13.207.62:41729 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 07:04:34.384757 27871 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:34.384826 27871 heartbeater.cc:511] Master 127.13.207.62:41729 requested a full tablet report, sending...
I20260808 07:04:34.385511 27871 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:34.385594 27871 heartbeater.cc:511] Master 127.13.207.62:41729 requested a full tablet report, sending...
W20260808 07:04:34.385972 27871 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:04:34.387719 27875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:34.376093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39038 (local address 127.13.207.62:41729)
0808 07:04:34.376241 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:34.376243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:34.378924 (+ 2681us) server_negotiation.cc:407] Connection header received
0808 07:04:34.379333 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:34.379335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:34.379355 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:34.379448 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:34.380122 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:34.380891 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:34.382599 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:34.382953 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:34.383027 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:34.384698 (+ 1671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:34.384700 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:34.384700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:34.384708 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:34.386364 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:34.386829 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:34.386830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:34.386831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:34.386901 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:34.387167 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:34.387168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:34.387168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:34.387300 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:34.387452 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:34.387572 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 07:04:34.387646 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":89,"threads_started":1}
I20260808 07:04:34.969146 27585 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:34.969408 27585 heartbeater.cc:511] Master 127.13.207.62:41729 requested a full tablet report, sending...
I20260808 07:04:34.969982 27375 ts_manager.cc:194] Registered new tserver with Master: 3c71b272049c41aeb6a6bc5f605007e1 (127.13.207.1:42773)
I20260808 07:04:34.970952 27375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:50655
Aug 08 07:04:34 dist-test-slave-8tdl krb5kdc[26333](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786172674, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:35.156847 27731 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:35.157074 27731 heartbeater.cc:511] Master 127.13.207.62:41729 requested a full tablet report, sending...
I20260808 07:04:35.157462 27375 ts_manager.cc:194] Registered new tserver with Master: a941acc5aaf4474c82c6c735e0abfa4a (127.13.207.2:40445)
I20260808 07:04:35.157876 27375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:57763
I20260808 07:04:35.386173 27871 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:35.386364 27871 heartbeater.cc:511] Master 127.13.207.62:41729 requested a full tablet report, sending...
I20260808 07:04:35.386735 27375 ts_manager.cc:194] Registered new tserver with Master: 29c544fac2f7423483a6055125a347ce (127.13.207.3:39831)
I20260808 07:04:35.387323 27375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:60799
I20260808 07:04:35.395797 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:04:35.407284 27895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:35.397142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39040 (local address 127.13.207.62:41729)
0808 07:04:35.397385 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:35.397388 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:35.397397 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:04:35.397423 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:35.397424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:35.397436 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:35.397518 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:35.398362 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.399185 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.400405 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.400731 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.400802 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:35.403118 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:35.403120 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:35.403121 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:35.403131 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:35.405218 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:35.405729 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:35.405730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:35.405730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:35.405806 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:35.406050 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:35.406051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:35.406051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:35.406304 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:35.406416 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:35.406994 (+ 578us) server_negotiation.cc:299] Negotiation successful
0808 07:04:35.407111 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":189,"threads_started":1}
I20260808 07:04:35.414767 27375 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39040:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:04:35.417634 27375 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:04:35.428571 27904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:35.422733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40048 (local address 127.13.207.3:39831)
0808 07:04:35.423115 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:35.423121 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:35.423595 (+ 474us) server_negotiation.cc:407] Connection header received
0808 07:04:35.423650 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:35.423653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:35.423672 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:35.423780 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:35.424120 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.425264 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.427026 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.427917 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.427981 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:35.428051 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:35.428405 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 07:04:35.428456 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":139,"threads_started":1}
I20260808 07:04:35.428870 27901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:35.422375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:39831 (local address 127.0.0.1:40048)
0808 07:04:35.423508 (+ 1133us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:35.423517 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:35.423579 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:35.423919 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:35.423923 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:35.423926 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:35.424012 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:35.424015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.425406 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.425407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:35.426899 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:35.426903 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.428130 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.428131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:35.428249 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:35.428268 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 07:04:35.428692 (+ 424us) client_negotiation.cc:240] Negotiation successful
0808 07:04:35.428775 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1120,"thread_start_us":181,"threads_started":1}
I20260808 07:04:35.430456 27801 tablet_service.cc:1467] Processing CreateTablet for tablet cc288f0197904d0daf0db1ca4f96cb93 (DEFAULT_TABLE table=test-table [id=9e40cf3b82494fcc81a168b351e9b543]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:35.430909 27801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc288f0197904d0daf0db1ca4f96cb93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:35.437453 27902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:35.422371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:40445 (local address 127.0.0.1:43524)
0808 07:04:35.423209 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:35.423257 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:35.423330 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:35.426617 (+ 3287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:35.426618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:35.426621 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:35.426719 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:35.426721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.428052 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.428053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:35.429199 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:35.429201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.437174 (+ 7973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.437176 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:35.437262 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:35.437268 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:04:35.437302 (+ 34us) client_negotiation.cc:240] Negotiation successful
0808 07:04:35.437357 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"thread_start_us":122,"threads_started":1}
I20260808 07:04:35.438297 27905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:35.422895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43524 (local address 127.13.207.2:40445)
0808 07:04:35.426404 (+ 3509us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:35.426409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:35.426422 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:04:35.426460 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:35.426462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:35.426480 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:35.426796 (+ 316us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:35.426879 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.427943 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.435979 (+ 8036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.437030 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.437917 (+ 887us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:35.438030 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:35.438064 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 07:04:35.438114 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3455,"thread_start_us":102,"threads_started":1}
I20260808 07:04:35.438851 27907 tablet_bootstrap.cc:492] T cc288f0197904d0daf0db1ca4f96cb93 P 29c544fac2f7423483a6055125a347ce: Bootstrap starting.
I20260808 07:04:35.439500 27659 tablet_service.cc:1467] Processing CreateTablet for tablet cc288f0197904d0daf0db1ca4f96cb93 (DEFAULT_TABLE table=test-table [id=9e40cf3b82494fcc81a168b351e9b543]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:35.439853 27907 tablet_bootstrap.cc:654] T cc288f0197904d0daf0db1ca4f96cb93 P 29c544fac2f7423483a6055125a347ce: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:35.440037 27659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc288f0197904d0daf0db1ca4f96cb93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:35.440191 27907 log.cc:824] T cc288f0197904d0daf0db1ca4f96cb93 P 29c544fac2f7423483a6055125a347ce: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:35.441121 27907 tablet_bootstrap.cc:492] T cc288f0197904d0daf0db1ca4f96cb93 P 29c544fac2f7423483a6055125a347ce: No bootstrap required, opened a new log
I20260808 07:04:35.441190 27907 ts_tablet_manager.cc:1397] T cc288f0197904d0daf0db1ca4f96cb93 P 29c544fac2f7423483a6055125a347ce: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:04:35.444088 27903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:35.422797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49356 (local address 127.13.207.1:42773)
0808 07:04:35.423461 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:35.423466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:35.429034 (+ 5568us) server_negotiation.cc:407] Connection header received
0808 07:04:35.429164 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:35.429167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:35.429186 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:35.429246 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:35.429506 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.430807 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.432208 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.433091 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.433143 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:35.433208 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:35.443867 (+ 10659us) server_negotiation.cc:299] Negotiation successful
0808 07:04:35.443949 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":71,"threads_started":1}
I20260808 07:04:35.443856 27907 raft_consensus.cc:348] T cc288f0197904d0daf0db1ca4f96cb93 P 29c544fac2f7423483a6055125a347ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a941acc5aaf4474c82c6c735e0abfa4a" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40445 } } peers { permanent_uuid: "29c544fac2f7423483a6055125a347ce" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 39831 } } peers { permanent_uuid: "3c71b272049c41aeb6a6bc5f605007e1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 42773 } }
I20260808 07:04:35.444169 27907 raft_consensus.cc:374] T cc288f0197904d0daf0db1ca4f96cb93 P 29c544fac2f7423483a6055125a347ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:35.444239 27907 raft_consensus.cc:729] T cc288f0197904d0daf0db1ca4f96cb93 P 29c544fac2f7423483a6055125a347ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c544fac2f7423483a6055125a347ce, State: Initialized, Role: FOLLOWER
I20260808 07:04:35.444384 27901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:35.426094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:42773 (local address 127.0.0.1:49356)
0808 07:04:35.428949 (+ 2855us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:35.428962 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:35.429070 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:35.429368 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:35.429369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:35.429371 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:35.429406 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:35.429407 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.430915 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.430916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:35.432078 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:35.432081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.443602 (+ 11521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.443603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:35.443710 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:35.443716 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:04:35.444220 (+ 504us) client_negotiation.cc:240] Negotiation successful
0808 07:04:35.444307 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2840,"thread_start_us":193,"threads_started":1}
I20260808 07:04:35.444984 27907 consensus_queue.cc:260] T cc288f0197904d0daf0db1ca4f96cb93 P 29c544fac2f7423483a6055125a347ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a941acc5aaf4474c82c6c735e0abfa4a" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40445 } } peers { permanent_uuid: "29c544fac2f7423483a6055125a347ce" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 39831 } } peers { permanent_uuid: "3c71b272049c41aeb6a6bc5f605007e1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 42773 } }
I20260808 07:04:35.445511 27907 ts_tablet_manager.cc:1428] T cc288f0197904d0daf0db1ca4f96cb93 P 29c544fac2f7423483a6055125a347ce: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 07:04:35.445470 27513 tablet_service.cc:1467] Processing CreateTablet for tablet cc288f0197904d0daf0db1ca4f96cb93 (DEFAULT_TABLE table=test-table [id=9e40cf3b82494fcc81a168b351e9b543]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:35.445693 27871 heartbeater.cc:503] Master 127.13.207.62:41729 was elected leader, sending a full tablet report...
I20260808 07:04:35.445871 27513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc288f0197904d0daf0db1ca4f96cb93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:35.455637 27910 tablet_bootstrap.cc:492] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a: Bootstrap starting.
I20260808 07:04:35.456693 27910 tablet_bootstrap.cc:654] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:35.457187 27910 log.cc:824] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:35.458240 27911 tablet_bootstrap.cc:492] T cc288f0197904d0daf0db1ca4f96cb93 P 3c71b272049c41aeb6a6bc5f605007e1: Bootstrap starting.
I20260808 07:04:35.459225 27911 tablet_bootstrap.cc:654] T cc288f0197904d0daf0db1ca4f96cb93 P 3c71b272049c41aeb6a6bc5f605007e1: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:35.459678 27911 log.cc:824] T cc288f0197904d0daf0db1ca4f96cb93 P 3c71b272049c41aeb6a6bc5f605007e1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:35.463054 27910 tablet_bootstrap.cc:492] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a: No bootstrap required, opened a new log
I20260808 07:04:35.463233 27910 ts_tablet_manager.cc:1397] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260808 07:04:35.465778 27910 raft_consensus.cc:348] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a941acc5aaf4474c82c6c735e0abfa4a" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40445 } } peers { permanent_uuid: "29c544fac2f7423483a6055125a347ce" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 39831 } } peers { permanent_uuid: "3c71b272049c41aeb6a6bc5f605007e1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 42773 } }
I20260808 07:04:35.466100 27910 raft_consensus.cc:374] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:35.466195 27910 raft_consensus.cc:729] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a941acc5aaf4474c82c6c735e0abfa4a, State: Initialized, Role: FOLLOWER
I20260808 07:04:35.466434 27910 consensus_queue.cc:260] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a941acc5aaf4474c82c6c735e0abfa4a" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40445 } } peers { permanent_uuid: "29c544fac2f7423483a6055125a347ce" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 39831 } } peers { permanent_uuid: "3c71b272049c41aeb6a6bc5f605007e1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 42773 } }
I20260808 07:04:35.466897 27910 ts_tablet_manager.cc:1428] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 07:04:35.467089 27731 heartbeater.cc:503] Master 127.13.207.62:41729 was elected leader, sending a full tablet report...
I20260808 07:04:35.468827 27911 tablet_bootstrap.cc:492] T cc288f0197904d0daf0db1ca4f96cb93 P 3c71b272049c41aeb6a6bc5f605007e1: No bootstrap required, opened a new log
I20260808 07:04:35.468921 27911 ts_tablet_manager.cc:1397] T cc288f0197904d0daf0db1ca4f96cb93 P 3c71b272049c41aeb6a6bc5f605007e1: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20260808 07:04:35.472363 27911 raft_consensus.cc:348] T cc288f0197904d0daf0db1ca4f96cb93 P 3c71b272049c41aeb6a6bc5f605007e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a941acc5aaf4474c82c6c735e0abfa4a" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40445 } } peers { permanent_uuid: "29c544fac2f7423483a6055125a347ce" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 39831 } } peers { permanent_uuid: "3c71b272049c41aeb6a6bc5f605007e1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 42773 } }
I20260808 07:04:35.472723 27911 raft_consensus.cc:374] T cc288f0197904d0daf0db1ca4f96cb93 P 3c71b272049c41aeb6a6bc5f605007e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:35.472839 27911 raft_consensus.cc:729] T cc288f0197904d0daf0db1ca4f96cb93 P 3c71b272049c41aeb6a6bc5f605007e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c71b272049c41aeb6a6bc5f605007e1, State: Initialized, Role: FOLLOWER
I20260808 07:04:35.473075 27911 consensus_queue.cc:260] T cc288f0197904d0daf0db1ca4f96cb93 P 3c71b272049c41aeb6a6bc5f605007e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a941acc5aaf4474c82c6c735e0abfa4a" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40445 } } peers { permanent_uuid: "29c544fac2f7423483a6055125a347ce" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 39831 } } peers { permanent_uuid: "3c71b272049c41aeb6a6bc5f605007e1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 42773 } }
I20260808 07:04:35.473526 27911 ts_tablet_manager.cc:1428] T cc288f0197904d0daf0db1ca4f96cb93 P 3c71b272049c41aeb6a6bc5f605007e1: Time spent starting tablet: real 0.005s user 0.003s sys 0.001s
I20260808 07:04:35.473711 27585 heartbeater.cc:503] Master 127.13.207.62:41729 was elected leader, sending a full tablet report...
Aug 08 07:04:35 dist-test-slave-8tdl krb5kdc[26333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172674, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for HTTP/127.13.207.20@KRBTEST.COM
W20260808 07:04:35.620594 27872 tablet.cc:2369] T cc288f0197904d0daf0db1ca4f96cb93 P 29c544fac2f7423483a6055125a347ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:04:35.623342 27914 raft_consensus.cc:482] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:04:35.623545 27914 raft_consensus.cc:504] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a941acc5aaf4474c82c6c735e0abfa4a" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40445 } } peers { permanent_uuid: "29c544fac2f7423483a6055125a347ce" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 39831 } } peers { permanent_uuid: "3c71b272049c41aeb6a6bc5f605007e1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 42773 } }
I20260808 07:04:35.624050 27914 leader_election.cc:288] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c544fac2f7423483a6055125a347ce (127.13.207.3:39831), 3c71b272049c41aeb6a6bc5f605007e1 (127.13.207.1:42773)
I20260808 07:04:35.628594 27904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:35.624776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:59081 (local address 127.13.207.3:39831)
0808 07:04:35.624839 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:35.624841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:35.624852 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:04:35.624883 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:35.624884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:35.624895 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:35.624987 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:35.625179 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.625916 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.627522 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.628100 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.628165 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:35.628175 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:35.628442 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 07:04:35.628509 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 07:04:35.629130 27917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:35.624258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:39831 (local address 127.13.207.2:59081)
0808 07:04:35.624500 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:35.624511 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:35.624587 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:35.624985 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:35.624986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:35.624988 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:35.625087 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:35.625089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.626052 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.626053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:35.627384 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:35.627389 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.628268 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.628269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:35.628350 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:35.628354 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:04:35.628971 (+ 617us) client_negotiation.cc:240] Negotiation successful
0808 07:04:35.629028 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":115,"threads_started":1}
I20260808 07:04:35.629737 27822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc288f0197904d0daf0db1ca4f96cb93" candidate_uuid: "a941acc5aaf4474c82c6c735e0abfa4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c544fac2f7423483a6055125a347ce" is_pre_election: true
I20260808 07:04:35.629823 27918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:35.624499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:42773 (local address 127.13.207.2:40227)
0808 07:04:35.624737 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:35.624748 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:35.624814 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:35.625042 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:35.625043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:35.625046 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:35.625115 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:35.625117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.627519 (+ 2402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.627521 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:35.628739 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:35.628743 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.629572 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.629573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:35.629644 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:35.629648 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:04:35.629704 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 07:04:35.629744 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":162,"threads_started":1}
I20260808 07:04:35.629865 27903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:35.624839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:40227 (local address 127.13.207.1:42773)
0808 07:04:35.624895 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:35.624897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:35.624905 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:04:35.624931 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:35.624932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:35.624941 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:35.624992 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:35.625190 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.627396 (+ 2206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.628863 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.629428 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.629480 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:35.629488 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:35.629742 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 07:04:35.629797 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 07:04:35.629946 27822 raft_consensus.cc:2454] T cc288f0197904d0daf0db1ca4f96cb93 P 29c544fac2f7423483a6055125a347ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a941acc5aaf4474c82c6c735e0abfa4a in term 0.
I20260808 07:04:35.630213 27610 leader_election.cc:302] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c544fac2f7423483a6055125a347ce, a941acc5aaf4474c82c6c735e0abfa4a; no voters:
I20260808 07:04:35.630198 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc288f0197904d0daf0db1ca4f96cb93" candidate_uuid: "a941acc5aaf4474c82c6c735e0abfa4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c71b272049c41aeb6a6bc5f605007e1" is_pre_election: true
I20260808 07:04:35.630354 27534 raft_consensus.cc:2454] T cc288f0197904d0daf0db1ca4f96cb93 P 3c71b272049c41aeb6a6bc5f605007e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a941acc5aaf4474c82c6c735e0abfa4a in term 0.
I20260808 07:04:35.630360 27914 raft_consensus.cc:2781] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:04:35.630409 27914 raft_consensus.cc:482] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:04:35.630434 27914 raft_consensus.cc:3037] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:35.631297 27914 raft_consensus.cc:504] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a941acc5aaf4474c82c6c735e0abfa4a" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40445 } } peers { permanent_uuid: "29c544fac2f7423483a6055125a347ce" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 39831 } } peers { permanent_uuid: "3c71b272049c41aeb6a6bc5f605007e1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 42773 } }
I20260808 07:04:35.631474 27914 leader_election.cc:288] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a [CANDIDATE]: Term 1 election: Requested vote from peers 29c544fac2f7423483a6055125a347ce (127.13.207.3:39831), 3c71b272049c41aeb6a6bc5f605007e1 (127.13.207.1:42773)
I20260808 07:04:35.631671 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc288f0197904d0daf0db1ca4f96cb93" candidate_uuid: "a941acc5aaf4474c82c6c735e0abfa4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c71b272049c41aeb6a6bc5f605007e1"
I20260808 07:04:35.631759 27822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc288f0197904d0daf0db1ca4f96cb93" candidate_uuid: "a941acc5aaf4474c82c6c735e0abfa4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c544fac2f7423483a6055125a347ce"
I20260808 07:04:35.631843 27822 raft_consensus.cc:3037] T cc288f0197904d0daf0db1ca4f96cb93 P 29c544fac2f7423483a6055125a347ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:35.631986 27534 raft_consensus.cc:3037] T cc288f0197904d0daf0db1ca4f96cb93 P 3c71b272049c41aeb6a6bc5f605007e1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:35.632866 27822 raft_consensus.cc:2454] T cc288f0197904d0daf0db1ca4f96cb93 P 29c544fac2f7423483a6055125a347ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a941acc5aaf4474c82c6c735e0abfa4a in term 1.
I20260808 07:04:35.632984 27534 raft_consensus.cc:2454] T cc288f0197904d0daf0db1ca4f96cb93 P 3c71b272049c41aeb6a6bc5f605007e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a941acc5aaf4474c82c6c735e0abfa4a in term 1.
I20260808 07:04:35.633145 27610 leader_election.cc:302] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c544fac2f7423483a6055125a347ce, a941acc5aaf4474c82c6c735e0abfa4a; no voters:
I20260808 07:04:35.633320 27914 raft_consensus.cc:2781] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:35.633553 27914 raft_consensus.cc:686] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a [term 1 LEADER]: Becoming Leader. State: Replica: a941acc5aaf4474c82c6c735e0abfa4a, State: Running, Role: LEADER
I20260808 07:04:35.633665 27914 consensus_queue.cc:237] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a941acc5aaf4474c82c6c735e0abfa4a" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40445 } } peers { permanent_uuid: "29c544fac2f7423483a6055125a347ce" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 39831 } } peers { permanent_uuid: "3c71b272049c41aeb6a6bc5f605007e1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 42773 } }
I20260808 07:04:35.634527 27375 catalog_manager.cc:5495] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a reported cstate change: term changed from 0 to 1, leader changed from <none> to a941acc5aaf4474c82c6c735e0abfa4a (127.13.207.2). New cstate: current_term: 1 leader_uuid: "a941acc5aaf4474c82c6c735e0abfa4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a941acc5aaf4474c82c6c735e0abfa4a" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c544fac2f7423483a6055125a347ce" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c71b272049c41aeb6a6bc5f605007e1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 42773 } health_report { overall_health: UNKNOWN } } }
W20260808 07:04:35.639032 27732 tablet.cc:2369] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:04:35.665704 27905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:35.663075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43540 (local address 127.13.207.2:40445)
0808 07:04:35.663120 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:35.663122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:35.663129 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:04:35.663179 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:35.663179 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:35.663201 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:35.663277 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:04:35.663412 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.663998 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.664821 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:35.665059 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:35.665155 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:35.665327 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:04:35.665452 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:04:35.665560 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:35.665578 (+ 18us) server_negotiation.cc:299] Negotiation successful
0808 07:04:35.665603 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:04:35.667843 27822 raft_consensus.cc:1261] T cc288f0197904d0daf0db1ca4f96cb93 P 29c544fac2f7423483a6055125a347ce [term 1 FOLLOWER]: Refusing update from remote peer a941acc5aaf4474c82c6c735e0abfa4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:35.667838 27534 raft_consensus.cc:1261] T cc288f0197904d0daf0db1ca4f96cb93 P 3c71b272049c41aeb6a6bc5f605007e1 [term 1 FOLLOWER]: Refusing update from remote peer a941acc5aaf4474c82c6c735e0abfa4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:35.668224 27919 consensus_queue.cc:1035] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c71b272049c41aeb6a6bc5f605007e1" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 42773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:04:35.668416 27919 consensus_queue.cc:1035] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c544fac2f7423483a6055125a347ce" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:04:35.673408 27923 mvcc.cc:205] Tried to move back new op lower bound from 7316163279532498944 to 7316163279396143104. Current Snapshot: MvccSnapshot[applied={T|T < 7316163279532498944}]
I20260808 07:04:35.673586 27924 mvcc.cc:205] Tried to move back new op lower bound from 7316163279532498944 to 7316163279396143104. Current Snapshot: MvccSnapshot[applied={T|T < 7316163279532498944}]
I20260808 07:04:35.674000 27922 mvcc.cc:205] Tried to move back new op lower bound from 7316163279532498944 to 7316163279396143104. Current Snapshot: MvccSnapshot[applied={T|T < 7316163279532498944}]
I20260808 07:04:35.677613 27375 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:04:35.677804 27375 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:04:35.678870 27375 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c94ae9d39cb643dba7f84091a9e70aa5: Sending DeleteTablet for 3 replicas of tablet cc288f0197904d0daf0db1ca4f96cb93
I20260808 07:04:35.679692 27659 tablet_service.cc:1514] Processing DeleteTablet for tablet cc288f0197904d0daf0db1ca4f96cb93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:35 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:43524
I20260808 07:04:35.680259 27513 tablet_service.cc:1514] Processing DeleteTablet for tablet cc288f0197904d0daf0db1ca4f96cb93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:35 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:49356
I20260808 07:04:35.681007 27801 tablet_service.cc:1514] Processing DeleteTablet for tablet cc288f0197904d0daf0db1ca4f96cb93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:35 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:40048
I20260808 07:04:35.682026 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 27414
I20260808 07:04:35.692124 27936 tablet_replica.cc:330] T cc288f0197904d0daf0db1ca4f96cb93 P 29c544fac2f7423483a6055125a347ce: stopping tablet replica
I20260808 07:04:35.692409 27936 raft_consensus.cc:2229] T cc288f0197904d0daf0db1ca4f96cb93 P 29c544fac2f7423483a6055125a347ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:04:35.692751 27936 raft_consensus.cc:2258] T cc288f0197904d0daf0db1ca4f96cb93 P 29c544fac2f7423483a6055125a347ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:04:35.693513 27936 ts_tablet_manager.cc:1905] T cc288f0197904d0daf0db1ca4f96cb93 P 29c544fac2f7423483a6055125a347ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:04:35.700573 27932 tablet_replica.cc:330] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a: stopping tablet replica
I20260808 07:04:35.700835 27932 raft_consensus.cc:2229] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:04:35.701073 27932 raft_consensus.cc:2258] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:04:35.701368 27936 ts_tablet_manager.cc:1918] T cc288f0197904d0daf0db1ca4f96cb93 P 29c544fac2f7423483a6055125a347ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:04:35.701540 27936 log.cc:1196] T cc288f0197904d0daf0db1ca4f96cb93 P 29c544fac2f7423483a6055125a347ce: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/cc288f0197904d0daf0db1ca4f96cb93
I20260808 07:04:35.701776 27932 ts_tablet_manager.cc:1905] T cc288f0197904d0daf0db1ca4f96cb93 P a941acc5aaf4474c82c6c735e0abfa4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:04:35.702039 27936 ts_tablet_manager.cc:1939] T cc288f0197904d0daf0db1ca4f96cb93 P 29c544fac2f7423483a6055125a347ce: Deleting consensus metadata
I20260808 07:04:35.702718 27359 catalog_manager.cc:4841] TS 29c544fac2f7423483a6055125a347ce (127.13.207.3:39831): tablet cc288f0197904d0daf0db1ca4f96cb93 (table test-table [id=9e40cf3b82494fcc81a168b351e9b543]) successfully deleted
W20260808 07:04:35.703192 27360 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.207.1:42773 (error 108)
I20260808 07:04:35.704047 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 27587
I20260808 07:04:35.711611 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 27734
I20260808 07:04:35.715222 27901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:35.715024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:42773 (local address 127.0.0.1:49368)
0808 07:04:35.715108 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:35.715170 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.1:42773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 07:04:35.715222 27902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:35.715131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:40445 (local address 127.0.0.1:43544)
0808 07:04:35.715173 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:35.715190 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.2:40445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 07:04:35.715673 27360 catalog_manager.cc:4584] TS 3c71b272049c41aeb6a6bc5f605007e1 (127.13.207.1:42773): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cc288f0197904d0daf0db1ca4f96cb93: Network error: Client connection negotiation failed: client connection to 127.13.207.1:42773: connect: Connection refused (error 111)
I20260808 07:04:35.719667 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 27340
I20260808 07:04:35.731310 14140 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 07:04:36.087137 14140 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 07:04:36.087247 14140 mini_ranger.cc:66] Stopping Ranger...
I20260808 07:04:36.443265 14140 mini_ranger.cc:68] Stopped Ranger
2026-08-08 07:04:36.443 UTC [26365] LOG: received smart shutdown request
2026-08-08 07:04:36.449 UTC [26365] LOG: background worker "logical replication launcher" (PID 26374) exited with exit code 1
2026-08-08 07:04:36.449 UTC [26369] LOG: shutting down
2026-08-08 07:04:36.689 UTC [26365] LOG: database system is shut down
2026-08-08T07:04:36Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (61180 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:36 dist-test-slave-8tdl krb5kdc[27946](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:36 dist-test-slave-8tdl krb5kdc[27946](info): set up 2 sockets
Aug 08 07:04:36 dist-test-slave-8tdl krb5kdc[27946](info): commencing operation
krb5kdc: starting...
W20260808 07:04:37.969152 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:04:38 dist-test-slave-8tdl krb5kdc[27946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:04:38.116045 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:39713
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:37251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:39713,127.13.207.61:44707,127.13.207.60:44011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:38.210253 27962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:38.210566 27962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:38.210613 27962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:38.212111 27962 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:38.212245 27962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:38.212272 27962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:38.212294 27962 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:38.212328 27962 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:38.214246 27962 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:37251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:39713,127.13.207.61:44707,127.13.207.60:44011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:39713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:38.214598 27962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:38.214922 27962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:38.220594 27969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:38.220738 27967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:38.220605 27972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:38.220605 27962 server_base.cc:1034] running on GCE node
I20260808 07:04:38.221566 27962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:38.221963 27962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:38.223133 27962 hybrid_clock.cc:648] HybridClock initialized: now 1786172678223105 us; error 45 us; skew 500 ppm
Aug 08 07:04:38 dist-test-slave-8tdl krb5kdc[27946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172678, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:38.226166 27962 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:04:38.226935 27962 webserver.cc:460] Webserver started at http://127.13.207.62:42091/ using document root <none> and password file <none>
I20260808 07:04:38.227206 27962 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:38.227257 27962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:38.227403 27962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:38.228572 27962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "d9b85f1f8f6241d5921288c2b423046e"
format_stamp: "Formatted at 2026-08-08 07:04:38 on dist-test-slave-8tdl"
server_key: "ff323d7b354a6cd11766dfc03045b97b"
server_key_iv: "0ffd0f2b237575c60a7d988d49c7a0c6"
server_key_version: "encryptionkey@0"
I20260808 07:04:38.228963 27962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "d9b85f1f8f6241d5921288c2b423046e"
format_stamp: "Formatted at 2026-08-08 07:04:38 on dist-test-slave-8tdl"
server_key: "ff323d7b354a6cd11766dfc03045b97b"
server_key_iv: "0ffd0f2b237575c60a7d988d49c7a0c6"
server_key_version: "encryptionkey@0"
I20260808 07:04:38.230463 27962 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:38.231545 27979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:38.231868 27962 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:38.232052 27962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "d9b85f1f8f6241d5921288c2b423046e"
format_stamp: "Formatted at 2026-08-08 07:04:38 on dist-test-slave-8tdl"
server_key: "ff323d7b354a6cd11766dfc03045b97b"
server_key_iv: "0ffd0f2b237575c60a7d988d49c7a0c6"
server_key_version: "encryptionkey@0"
I20260808 07:04:38.232146 27962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:38.248775 27962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:38.251264 27962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:38.251462 27962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:38.256098 27962 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:39713
I20260808 07:04:38.256150 28037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:39713 every 8 connection(s)
I20260808 07:04:38.256539 27962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:38.257467 28038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:38.259205 28038 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39713 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:04:38.263305 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 27962
I20260808 07:04:38.263429 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:04:38.263665 14140 external_mini_cluster.cc:1428] Setting key d51817511f6046fb3d4cf5ea1a6f9351
Aug 08 07:04:38 dist-test-slave-8tdl krb5kdc[27946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172678, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:04:38.270366 28041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.259912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:39713 (local address 127.0.0.1:59192)
0808 07:04:38.260102 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.260146 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:38.260633 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:38.261131 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:38.261134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:38.261444 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:38.261615 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:38.261619 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.262829 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.262830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:38.263632 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:38.263634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.264193 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.264194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:38.264269 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.264877 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:38.264886 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:38.266360 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:38.268571 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:38.268580 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:38.268583 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:38.268883 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:38.269135 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:38.269136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:38.269136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:38.269259 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:38.269650 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:38.269658 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:38.269745 (+ 87us) client_negotiation.cc:769] Sending connection context
0808 07:04:38.269867 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 07:04:38.270087 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":105,"threads_started":1}
I20260808 07:04:38.270401 28042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.259965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59192 (local address 127.13.207.62:39713)
0808 07:04:38.260152 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:38.260157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:38.260223 (+ 66us) server_negotiation.cc:407] Connection header received
0808 07:04:38.260790 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:38.260792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:38.260810 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:38.260929 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:38.261720 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.262657 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.263726 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.264090 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.264231 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.266487 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:38.266495 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:38.266496 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:38.266503 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:38.268470 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.268972 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.268975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.268976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.269062 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.269332 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.269333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.269333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.269577 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:38.269646 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:38.269947 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 07:04:38.270153 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":103,"server-negotiator.queue_time_us":154,"thread_start_us":128,"threads_started":1}
I20260808 07:04:38.271206 28038 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:04:38.271610 28041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.271472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:44707 (local address 127.0.0.1:56376)
0808 07:04:38.271526 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.271578 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.61:44707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 07:04:38.271771 27982 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.207.61:44707: connect: Connection refused (error 111)
I20260808 07:04:38.272269 28041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.272154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:44707 (local address 127.0.0.1:56378)
0808 07:04:38.272204 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.272236 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.61:44707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 07:04:38.272430 28038 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.61:44707: Network error: Client connection negotiation failed: client connection to 127.13.207.61:44707: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.13.207.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.61.keytab.
Entry for principal kudu/127.13.207.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.61.keytab.
I20260808 07:04:38.295132 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.61:44707
--webserver_interface=127.13.207.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.61.keytab
--principal=kudu/127.13.207.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:37251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:39713,127.13.207.61:44707,127.13.207.60:44011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:04:38.321705 28038 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } attempt: 1
I20260808 07:04:38.322350 28041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.322186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:44707 (local address 127.0.0.1:56388)
0808 07:04:38.322253 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.322307 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.61:44707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 07:04:38.322825 28041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.322722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:44707 (local address 127.0.0.1:56396)
0808 07:04:38.322762 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.322800 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.61:44707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260808 07:04:38.323010 28038 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.61:44707: Network error: Client connection negotiation failed: client connection to 127.13.207.61:44707: connect: Connection refused (error 111)
W20260808 07:04:38.390954 28046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:38.391209 28046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:38.391237 28046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 07:04:38.391216 28038 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } attempt: 2
I20260808 07:04:38.391769 28041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.391597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:44707 (local address 127.0.0.1:56402)
0808 07:04:38.391685 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.391732 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.61:44707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 07:04:38.392293 28041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.392164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:44707 (local address 127.0.0.1:56404)
0808 07:04:38.392233 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.392266 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.61:44707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 07:04:38.392433 28038 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.61:44707: Network error: Client connection negotiation failed: client connection to 127.13.207.61:44707: connect: Connection refused (error 111)
W20260808 07:04:38.393029 28046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:38.393098 28046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:38.393136 28046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:38.393169 28046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:38.393201 28046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:38.395128 28046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:37251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:39713,127.13.207.61:44707,127.13.207.60:44011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.61.keytab
--principal=kudu/127.13.207.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.61:44707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.207.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:38.395475 28046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:38.395740 28046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:38.401700 28052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:38.401714 28046 server_base.cc:1034] running on GCE node
W20260808 07:04:38.401679 28051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:38.401679 28055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:38.402374 28046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:38.402812 28046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:38.403999 28046 hybrid_clock.cc:648] HybridClock initialized: now 1786172678403984 us; error 33 us; skew 500 ppm
Aug 08 07:04:38 dist-test-slave-8tdl krb5kdc[27946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172678, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:38.407486 28046 init.cc:377] Logged in from keytab as kudu/127.13.207.61@KRBTEST.COM (short username kudu)
I20260808 07:04:38.408013 28046 webserver.cc:460] Webserver started at http://127.13.207.61:44661/ using document root <none> and password file <none>
I20260808 07:04:38.408267 28046 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:38.408337 28046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:38.408493 28046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:38.409638 28046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-1/data/instance:
uuid: "cbfcca3fc2f147759da30e04c54309c2"
format_stamp: "Formatted at 2026-08-08 07:04:38 on dist-test-slave-8tdl"
server_key: "26340f038a9c5104c37b6fff86abb209"
server_key_iv: "215d7d9cc8bb27ebbbc2e7fa187c45bc"
server_key_version: "encryptionkey@0"
I20260808 07:04:38.410027 28046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-1/wal/instance:
uuid: "cbfcca3fc2f147759da30e04c54309c2"
format_stamp: "Formatted at 2026-08-08 07:04:38 on dist-test-slave-8tdl"
server_key: "26340f038a9c5104c37b6fff86abb209"
server_key_iv: "215d7d9cc8bb27ebbbc2e7fa187c45bc"
server_key_version: "encryptionkey@0"
I20260808 07:04:38.411701 28046 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:04:38.412691 28063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:38.412964 28046 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:04:38.413084 28046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-1/wal
uuid: "cbfcca3fc2f147759da30e04c54309c2"
format_stamp: "Formatted at 2026-08-08 07:04:38 on dist-test-slave-8tdl"
server_key: "26340f038a9c5104c37b6fff86abb209"
server_key_iv: "215d7d9cc8bb27ebbbc2e7fa187c45bc"
server_key_version: "encryptionkey@0"
I20260808 07:04:38.413168 28046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:38.424784 28046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:38.427191 28046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:38.427368 28046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:38.431989 28046 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.61:44707
I20260808 07:04:38.432039 28119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.61:44707 every 8 connection(s)
I20260808 07:04:38.432447 28046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-1/data/info.pb
I20260808 07:04:38.433457 28120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:38.435316 28120 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39713 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:04:38.442133 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 28046
Aug 08 07:04:38 dist-test-slave-8tdl krb5kdc[27946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172678, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.61@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:04:38.442289 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-1/wal/instance
I20260808 07:04:38.442533 14140 external_mini_cluster.cc:1428] Setting key 0c1e2529a0b67b2ee95145d5ac819823
I20260808 07:04:38.446765 28042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.436245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59202 (local address 127.13.207.62:39713)
0808 07:04:38.436335 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:38.436337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:38.436598 (+ 261us) server_negotiation.cc:407] Connection header received
0808 07:04:38.437322 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:38.437323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:38.437339 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:38.437419 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:38.438209 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.438780 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.439985 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.440239 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.440309 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.443092 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:38.443093 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:38.443093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:38.443101 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:38.444937 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.445444 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.445446 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.445446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.445524 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.445936 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.445937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.445937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.446076 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:38.446136 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:38.446545 (+ 409us) server_negotiation.cc:299] Negotiation successful
0808 07:04:38.446627 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 07:04:38.447194 28122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.436212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:39713 (local address 127.0.0.1:59202)
0808 07:04:38.436408 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.436454 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:38.437049 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:38.437560 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:38.437564 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:38.437859 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:38.438083 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:38.438087 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.438875 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.438876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:38.439885 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:38.439888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.440346 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.440347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:38.440467 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.441075 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:38.441084 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:38.442959 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:38.445079 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:38.445089 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:38.445091 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:38.445361 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:38.445630 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:38.445632 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:38.445632 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:38.445853 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:38.446153 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:38.446162 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:38.446387 (+ 225us) client_negotiation.cc:769] Sending connection context
0808 07:04:38.446522 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 07:04:38.446802 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":107,"threads_started":1}
I20260808 07:04:38.448194 28120 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.207.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.60@KRBTEST.COM" created.
Aug 08 07:04:38 dist-test-slave-8tdl krb5kdc[27946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172678, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.61@KRBTEST.COM for kudu/127.13.207.61@KRBTEST.COM
I20260808 07:04:38.461252 28124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.448643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56418 (local address 127.13.207.61:44707)
0808 07:04:38.448842 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:38.448846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:38.448865 (+ 19us) server_negotiation.cc:407] Connection header received
0808 07:04:38.448882 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:38.448884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:38.448895 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:38.448979 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:38.449486 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.450276 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.451161 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.451515 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.451607 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.457823 (+ 6216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:38.457845 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:38.457846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:38.457855 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:38.459868 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.460308 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.460312 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.460313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.460373 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.460582 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.460583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.460583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.460763 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:38.460867 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:38.460989 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 07:04:38.461103 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":103,"threads_started":1}
I20260808 07:04:38.461261 28122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.448557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:44707 (local address 127.0.0.1:56418)
0808 07:04:38.448611 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.448623 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:38.448663 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:38.448986 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:38.448987 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:38.449147 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:38.449375 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:38.449378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.450390 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.450391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:38.451044 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:38.451047 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.451602 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.451604 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:38.451685 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.452187 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:38.452194 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:38.457691 (+ 5497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:38.460007 (+ 2316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:38.460008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:38.460009 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:38.460203 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:38.460447 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:38.460448 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:38.460448 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:38.460515 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:38.460839 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:38.460840 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:38.460883 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 07:04:38.460958 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 07:04:38.461161 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":96}
I20260808 07:04:38.461961 28120 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:04:38.462355 28122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.462222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:44011 (local address 127.0.0.1:57706)
0808 07:04:38.462274 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.462332 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:44011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 07:04:38.462507 28064 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.207.60:44011: connect: Connection refused (error 111)
I20260808 07:04:38.462989 28122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.462892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:44011 (local address 127.0.0.1:57722)
0808 07:04:38.462938 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.462965 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:44011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 07:04:38.463105 28120 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.60:44011: Network error: Client connection negotiation failed: client connection to 127.13.207.60:44011: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.60.keytab.
Entry for principal kudu/127.13.207.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.60.keytab.
I20260808 07:04:38.479522 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.60:44011
--webserver_interface=127.13.207.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.60.keytab
--principal=kudu/127.13.207.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:37251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:39713,127.13.207.61:44707,127.13.207.60:44011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:04:38.483603 28038 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } attempt: 3
Aug 08 07:04:38 dist-test-slave-8tdl krb5kdc[27946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172678, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for kudu/127.13.207.61@KRBTEST.COM
I20260808 07:04:38.492434 28041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.484042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:44707 (local address 127.0.0.1:56434)
0808 07:04:38.484128 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.484141 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:38.484223 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:38.484422 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:38.484423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:38.484597 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:38.484665 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:38.484667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.485457 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.485459 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:38.485976 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:38.485979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.486402 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.486403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:38.486486 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.487014 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:38.487019 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:38.488811 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:38.491128 (+ 2317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:38.491129 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:38.491129 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:38.491362 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:38.491650 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:38.491651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:38.491651 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:38.491729 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:38.492099 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:38.492100 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:38.492159 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 07:04:38.492239 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:04:38.492349 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 07:04:38.492548 28124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.484127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56434 (local address 127.13.207.61:44707)
0808 07:04:38.484197 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:38.484199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:38.484212 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:04:38.484315 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:38.484317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:38.484331 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:38.484413 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:38.484749 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.485280 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.486072 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.486300 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.486398 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.488929 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:38.488932 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:38.488932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:38.488943 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:38.490994 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.491444 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.491445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.491446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.491517 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.491809 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.491810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.491811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.491990 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:38.492074 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:38.492287 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 07:04:38.492400 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:04:38.493129 28038 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:04:38.493577 28041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.493418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:44011 (local address 127.0.0.1:57730)
0808 07:04:38.493506 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.493545 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:44011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 07:04:38.493975 28041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.493884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:44011 (local address 127.0.0.1:57742)
0808 07:04:38.493930 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.493956 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:44011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 07:04:38.494153 28038 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.60:44011: Network error: Client connection negotiation failed: client connection to 127.13.207.60:44011: connect: Connection refused (error 111)
I20260808 07:04:38.512358 28120 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } attempt: 1
I20260808 07:04:38.513360 28122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.513198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:44011 (local address 127.0.0.1:57758)
0808 07:04:38.513277 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.513323 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:44011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 07:04:38.513761 28122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.513653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:44011 (local address 127.0.0.1:57762)
0808 07:04:38.513706 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.513740 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:44011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 07:04:38.513861 28120 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.60:44011: Network error: Client connection negotiation failed: client connection to 127.13.207.60:44011: connect: Connection refused (error 111)
I20260808 07:04:38.525413 28038 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } attempt: 1
I20260808 07:04:38.525979 28041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.525819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:44011 (local address 127.0.0.1:57772)
0808 07:04:38.525899 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.525945 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:44011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 07:04:38.526400 28041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.526286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:44011 (local address 127.0.0.1:57778)
0808 07:04:38.526337 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.526379 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:44011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 07:04:38.526501 28038 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.60:44011: Network error: Client connection negotiation failed: client connection to 127.13.207.60:44011: connect: Connection refused (error 111)
W20260808 07:04:38.581437 28127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:38.581749 28127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:38.581799 28127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 07:04:38.582026 28120 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } attempt: 2
I20260808 07:04:38.582585 28122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.582436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:44011 (local address 127.0.0.1:57786)
0808 07:04:38.582512 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.582552 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:44011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 07:04:38.582983 28122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.582870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:44011 (local address 127.0.0.1:57800)
0808 07:04:38.582935 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.582966 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:44011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 07:04:38.583073 28120 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.60:44011: Network error: Client connection negotiation failed: client connection to 127.13.207.60:44011: connect: Connection refused (error 111)
W20260808 07:04:38.583523 28127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:38.583571 28127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:38.583588 28127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:38.583604 28127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:38.583618 28127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:38.585541 28127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:37251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:39713,127.13.207.61:44707,127.13.207.60:44011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.60.keytab
--principal=kudu/127.13.207.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.60:44011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.207.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:38.585870 28127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:38.586203 28127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:38.592549 28132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:38.592548 28135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:38.592684 28127 server_base.cc:1034] running on GCE node
W20260808 07:04:38.592557 28133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:38.593098 28127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:38.593464 28127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:38.594730 28127 hybrid_clock.cc:648] HybridClock initialized: now 1786172678594660 us; error 53 us; skew 500 ppm
Aug 08 07:04:38 dist-test-slave-8tdl krb5kdc[27946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172678, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:38.597844 28127 init.cc:377] Logged in from keytab as kudu/127.13.207.60@KRBTEST.COM (short username kudu)
I20260808 07:04:38.598399 28127 webserver.cc:460] Webserver started at http://127.13.207.60:45579/ using document root <none> and password file <none>
I20260808 07:04:38.598654 28127 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:38.598763 28127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:38.598941 28127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:38.600119 28127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-2/data/instance:
uuid: "bcc13d5c4f584051b8f76106c53164ba"
format_stamp: "Formatted at 2026-08-08 07:04:38 on dist-test-slave-8tdl"
server_key: "d535241bbdb7314a4fb2bdfe72580afe"
server_key_iv: "5cc6c69401053b1c0acb18e8aee7d4f0"
server_key_version: "encryptionkey@0"
I20260808 07:04:38.600517 28127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-2/wal/instance:
uuid: "bcc13d5c4f584051b8f76106c53164ba"
format_stamp: "Formatted at 2026-08-08 07:04:38 on dist-test-slave-8tdl"
server_key: "d535241bbdb7314a4fb2bdfe72580afe"
server_key_iv: "5cc6c69401053b1c0acb18e8aee7d4f0"
server_key_version: "encryptionkey@0"
I20260808 07:04:38.601662 28038 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } attempt: 2
I20260808 07:04:38.602151 28127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:38.602205 28041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.602051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:44011 (local address 127.0.0.1:57804)
0808 07:04:38.602121 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.602169 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:44011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 07:04:38.602670 28041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.602535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:44011 (local address 127.0.0.1:57810)
0808 07:04:38.602604 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.602639 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:44011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 07:04:38.602797 28038 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.60:44011: Network error: Client connection negotiation failed: client connection to 127.13.207.60:44011: connect: Connection refused (error 111)
I20260808 07:04:38.603164 28142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:38.603334 28127 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:04:38.603432 28127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-2/wal
uuid: "bcc13d5c4f584051b8f76106c53164ba"
format_stamp: "Formatted at 2026-08-08 07:04:38 on dist-test-slave-8tdl"
server_key: "d535241bbdb7314a4fb2bdfe72580afe"
server_key_iv: "5cc6c69401053b1c0acb18e8aee7d4f0"
server_key_version: "encryptionkey@0"
I20260808 07:04:38.603494 28127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:38.620339 28127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:38.623030 28127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:38.623206 28127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:38.627655 28127 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.60:44011
I20260808 07:04:38.628082 28127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-2/data/info.pb
I20260808 07:04:38.628028 28197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.60:44011 every 8 connection(s)
I20260808 07:04:38.628898 28198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:38.630652 28198 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39713 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:04:38.637023 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 28127
I20260808 07:04:38.637141 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172493058068-14140-0/minicluster-data/master-2/wal/instance
I20260808 07:04:38.637363 14140 external_mini_cluster.cc:1428] Setting key ff1f0e31979d1b60659897d4587220d4
Aug 08 07:04:38 dist-test-slave-8tdl krb5kdc[27946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172678, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.60@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:04:38.642808 28042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.631954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59206 (local address 127.13.207.62:39713)
0808 07:04:38.632024 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:38.632026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:38.632116 (+ 90us) server_negotiation.cc:407] Connection header received
0808 07:04:38.632865 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:38.632867 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:38.632880 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:38.632946 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:38.633643 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.634136 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.635782 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.636026 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.636097 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.639214 (+ 3117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:38.639216 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:38.639216 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:38.639224 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:38.640952 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.641460 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.641462 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.641462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.641536 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.641886 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.641887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.641887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.642071 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:38.642156 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:38.642555 (+ 399us) server_negotiation.cc:299] Negotiation successful
0808 07:04:38.642660 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:04:38.642947 28200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.631787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:39713 (local address 127.0.0.1:59206)
0808 07:04:38.631981 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.632033 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:38.632712 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:38.633026 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:38.633030 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:38.633306 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:38.633531 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:38.633535 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.634259 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.634259 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:38.635693 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:38.635696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.636124 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.636125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:38.636245 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.636972 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:38.636985 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:38.639098 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:38.641074 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:38.641084 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:38.641087 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:38.641365 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:38.641626 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:38.641627 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:38.641628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:38.641799 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:38.642230 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:38.642237 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:38.642366 (+ 129us) client_negotiation.cc:769] Sending connection context
0808 07:04:38.642477 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 07:04:38.642613 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":98,"threads_started":1}
Aug 08 07:04:38 dist-test-slave-8tdl krb5kdc[27946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.60@KRBTEST.COM
I20260808 07:04:38.643841 28198 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:04:38.648221 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.637830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57822 (local address 127.13.207.60:44011)
0808 07:04:38.637996 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:38.638001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:38.638128 (+ 127us) server_negotiation.cc:407] Connection header received
0808 07:04:38.638160 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:38.638162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:38.638189 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:38.638290 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:38.639027 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.640147 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.640939 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.641311 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.641403 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.644229 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:38.644239 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:38.644239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:38.644247 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:38.646243 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.646698 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.646702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.646702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.646791 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.647108 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.647110 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.647110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.647430 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:38.647606 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:38.647968 (+ 362us) server_negotiation.cc:299] Negotiation successful
0808 07:04:38.648125 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":85,"threads_started":1}
Aug 08 07:04:38 dist-test-slave-8tdl krb5kdc[27946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172678, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.60@KRBTEST.COM for kudu/127.13.207.61@KRBTEST.COM
I20260808 07:04:38.653201 28200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.644176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:44707 (local address 127.0.0.1:56440)
0808 07:04:38.644216 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.644226 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:38.644262 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:38.644494 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:38.644495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:38.644677 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:38.644746 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:38.644748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.645736 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.645736 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:38.646231 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:38.646233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.646754 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.646755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:38.646872 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.647515 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:38.647522 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:38.649223 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:38.651901 (+ 2678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:38.651903 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:38.651903 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:38.652151 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:38.652411 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:38.652412 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:38.652412 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:38.652489 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:38.652767 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:38.652768 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:38.652828 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 07:04:38.652911 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 07:04:38.653043 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260808 07:04:38.653268 28124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.644283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56440 (local address 127.13.207.61:44707)
0808 07:04:38.644353 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:38.644356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:38.644366 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:04:38.644395 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:38.644396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:38.644408 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:38.644490 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:38.644834 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.645635 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.646339 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.646596 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.646728 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.649326 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:38.649327 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:38.649328 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:38.649337 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:38.651775 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.652257 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.652258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.652258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.652324 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.652569 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.652570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.652570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.652694 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:38.652756 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:38.653068 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 07:04:38.653172 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Aug 08 07:04:38 dist-test-slave-8tdl krb5kdc[27946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:04:38.654266 28198 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:04:38 dist-test-slave-8tdl krb5kdc[27946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172678, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.60@KRBTEST.COM for kudu/127.13.207.60@KRBTEST.COM
I20260808 07:04:38.661696 28042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.649423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59218 (local address 127.13.207.62:39713)
0808 07:04:38.649494 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:38.649497 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:38.649505 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:04:38.649529 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:38.649530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:38.649541 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:38.649615 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:38.650102 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.650580 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.651308 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.651531 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.651611 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.654622 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:38.654623 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:38.654624 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:38.654634 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:38.659496 (+ 4862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.660278 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.660280 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.660280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.660373 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.660701 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.660703 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.660704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.660872 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:38.661104 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:38.661386 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 07:04:38.661512 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:04:38.664444 28200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.654635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:44011 (local address 127.0.0.1:57828)
0808 07:04:38.654735 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.654748 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:38.654805 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:38.655417 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:38.655418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:38.655611 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:38.655692 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:38.655694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.656719 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.656720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:38.657659 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:38.657663 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.658287 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.658287 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:38.658344 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.658954 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:38.658965 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:38.660878 (+ 1913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:38.663212 (+ 2334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:38.663213 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:38.663213 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:38.663439 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:38.663687 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:38.663706 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:38.663706 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:38.663778 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:38.664094 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:38.664095 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:38.664159 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 07:04:38.664221 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 07:04:38.664329 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260808 07:04:38.664733 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.655163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57828 (local address 127.13.207.60:44011)
0808 07:04:38.655215 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:38.655217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:38.655227 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:04:38.655273 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:38.655274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:38.655289 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:38.655356 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:38.655813 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.656620 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.657774 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.658158 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.658231 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.660981 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:38.660982 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:38.660983 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:38.660992 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:38.663125 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.663519 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.663521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.663526 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.663593 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.663853 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.663853 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.663854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.664021 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:38.664094 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:38.664584 (+ 490us) server_negotiation.cc:299] Negotiation successful
0808 07:04:38.664664 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:04:38.666909 28198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba: Bootstrap starting.
I20260808 07:04:38.667863 28198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:38.668496 28198 log.cc:824] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba: Log is configured to *not* fsync() on all Append() calls
Aug 08 07:04:38 dist-test-slave-8tdl krb5kdc[27946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.61@KRBTEST.COM
I20260808 07:04:38.669530 28198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba: No bootstrap required, opened a new log
I20260808 07:04:38.672601 28198 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b85f1f8f6241d5921288c2b423046e" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39713 } } peers { permanent_uuid: "cbfcca3fc2f147759da30e04c54309c2" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } } peers { permanent_uuid: "bcc13d5c4f584051b8f76106c53164ba" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } }
I20260808 07:04:38.672808 28198 raft_consensus.cc:374] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:38.672855 28198 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcc13d5c4f584051b8f76106c53164ba, State: Initialized, Role: FOLLOWER
I20260808 07:04:38.673007 28198 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b85f1f8f6241d5921288c2b423046e" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39713 } } peers { permanent_uuid: "cbfcca3fc2f147759da30e04c54309c2" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } } peers { permanent_uuid: "bcc13d5c4f584051b8f76106c53164ba" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } }
I20260808 07:04:38.673460 28198 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:38.673475 28204 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b85f1f8f6241d5921288c2b423046e" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39713 } } peers { permanent_uuid: "cbfcca3fc2f147759da30e04c54309c2" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } } peers { permanent_uuid: "bcc13d5c4f584051b8f76106c53164ba" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } } }
I20260808 07:04:38.673575 28124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.663355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56442 (local address 127.13.207.61:44707)
0808 07:04:38.663406 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:38.663408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:38.663418 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:04:38.663444 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:38.663445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:38.663456 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:38.663512 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:38.664450 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.665099 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.666056 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.666340 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.666433 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.669764 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:38.669766 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:38.669766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:38.669778 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:38.671986 (+ 2208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.672462 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.672471 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.672472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.672548 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.672804 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.672805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.672805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.672968 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:38.673180 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:38.673357 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 07:04:38.673455 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:04:38.673863 28204 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:04:38.674212 28120 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } attempt: 3
I20260808 07:04:38.675661 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 07:04:38.675798 28215 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:04:38.676252 28215 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 08 07:04:38 dist-test-slave-8tdl krb5kdc[27946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172678, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.61@KRBTEST.COM for kudu/127.13.207.60@KRBTEST.COM
I20260808 07:04:38.676805 28127 master_runner.cc:186] Error getting master registration for 127.13.207.62:39713: OK, instance_id { permanent_uuid: "d9b85f1f8f6241d5921288c2b423046e" instance_seqno: 1786172678254715 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:04:38.679463 28127 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:04:38.682983 28122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.674488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:44011 (local address 127.0.0.1:57842)
0808 07:04:38.674522 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.674533 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:38.674577 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:38.674848 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:38.674849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:38.675055 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:38.675105 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:38.675107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.676001 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.676002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:38.676754 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:38.676757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.677260 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.677261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:38.677322 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.677817 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:38.677823 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:38.679213 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:38.681087 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:38.681088 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:38.681089 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:38.681328 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:38.681585 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:38.681586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:38.681587 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:38.681837 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:38.682240 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:38.682241 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:38.682297 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 07:04:38.682574 (+ 277us) client_negotiation.cc:240] Negotiation successful
0808 07:04:38.682780 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15,"mutex_wait_us":4}
I20260808 07:04:38.683108 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.674546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57842 (local address 127.13.207.60:44011)
0808 07:04:38.674655 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:38.674657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:38.674663 (+ 6us) server_negotiation.cc:407] Connection header received
0808 07:04:38.674733 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:38.674733 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:38.674743 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:38.674838 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:38.675192 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.675921 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.676849 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.677139 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.677203 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.679309 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:38.679311 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:38.679311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:38.679319 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:38.680989 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.681411 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.681412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.681412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.681475 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.681938 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.681939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.681940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.682139 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:38.682214 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:38.682789 (+ 575us) server_negotiation.cc:299] Negotiation successful
0808 07:04:38.683003 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 07:04:38.685799 28120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2: Bootstrap starting.
I20260808 07:04:38.686960 28120 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:38.687469 28120 log.cc:824] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:38.687876 28124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.677223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56444 (local address 127.13.207.61:44707)
0808 07:04:38.677265 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:38.677267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:38.680037 (+ 2770us) server_negotiation.cc:407] Connection header received
0808 07:04:38.680073 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:38.680073 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:38.680085 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:38.680147 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:38.680907 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.681514 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.682554 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.682947 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.683025 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.684616 (+ 1591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:38.684618 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:38.684618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:38.684627 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:38.686316 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.686811 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.686813 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.686813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.686899 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.687158 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.687159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.687159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.687321 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:38.687410 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:38.687676 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 07:04:38.687769 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:04:38.688992 28120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2: No bootstrap required, opened a new log
I20260808 07:04:38.691642 28120 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b85f1f8f6241d5921288c2b423046e" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39713 } } peers { permanent_uuid: "cbfcca3fc2f147759da30e04c54309c2" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } } peers { permanent_uuid: "bcc13d5c4f584051b8f76106c53164ba" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } }
I20260808 07:04:38.691823 28120 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:38.691864 28120 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbfcca3fc2f147759da30e04c54309c2, State: Initialized, Role: FOLLOWER
I20260808 07:04:38.692024 28120 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b85f1f8f6241d5921288c2b423046e" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39713 } } peers { permanent_uuid: "cbfcca3fc2f147759da30e04c54309c2" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } } peers { permanent_uuid: "bcc13d5c4f584051b8f76106c53164ba" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } }
I20260808 07:04:38.692433 28120 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:38.692592 28225 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b85f1f8f6241d5921288c2b423046e" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39713 } } peers { permanent_uuid: "cbfcca3fc2f147759da30e04c54309c2" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } } peers { permanent_uuid: "bcc13d5c4f584051b8f76106c53164ba" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } } }
I20260808 07:04:38.693097 28225 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:04:38.695951 28042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.679807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59228 (local address 127.13.207.62:39713)
0808 07:04:38.679863 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:38.679865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:38.686892 (+ 7027us) server_negotiation.cc:407] Connection header received
0808 07:04:38.687022 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:38.687025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:38.687040 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:38.687075 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:38.687844 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.688689 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.689829 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.690176 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.690229 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.692170 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:38.692172 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:38.692172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:38.692180 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:38.694033 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.694456 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.694457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.694458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.694529 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.694793 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.694793 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.694794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.695233 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:38.695335 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:38.695757 (+ 422us) server_negotiation.cc:299] Negotiation successful
0808 07:04:38.695848 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260808 07:04:38.695853 28237 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:04:38.696566 28237 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:04:38.697770 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.686956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57848 (local address 127.13.207.60:44011)
0808 07:04:38.687003 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:38.687005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:38.687926 (+ 921us) server_negotiation.cc:407] Connection header received
0808 07:04:38.688595 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:38.688597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:38.688611 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:38.688642 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:38.689452 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.690200 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.691254 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.691543 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.691588 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.693748 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:38.693750 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:38.693751 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:38.693760 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:38.696081 (+ 2321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.696823 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.696825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.696826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.696889 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.697111 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.697111 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.697112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.697266 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:38.697340 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:38.697578 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 07:04:38.697670 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:04:38.697129 28046 master_runner.cc:186] Error getting master registration for 127.13.207.62:39713: OK, instance_id { permanent_uuid: "d9b85f1f8f6241d5921288c2b423046e" instance_seqno: 1786172678254715 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:04:38.700536 28046 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:04:38.702075 28038 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } attempt: 3
Aug 08 07:04:38 dist-test-slave-8tdl krb5kdc[27946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172678, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for kudu/127.13.207.60@KRBTEST.COM
I20260808 07:04:38.712025 28041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.702392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:44011 (local address 127.0.0.1:57864)
0808 07:04:38.702484 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:38.702499 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:38.702557 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:38.702943 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:38.702944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:38.703151 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:38.703212 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:38.703215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.704180 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.704181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:38.704966 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:38.704969 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.705468 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.705469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:38.705533 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.706187 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:38.706193 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:38.707901 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:38.710720 (+ 2819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:38.710721 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:38.710722 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:38.710979 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:38.711246 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:38.711246 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:38.711247 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:38.711326 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:38.711642 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:38.711643 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:38.711704 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 07:04:38.711781 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 07:04:38.711899 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 07:04:38.712018 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:38.702646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57864 (local address 127.13.207.60:44011)
0808 07:04:38.702796 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:38.702798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:38.702808 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:04:38.702835 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:38.702836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:38.702847 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:38.702913 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:38.703304 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.704084 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.705070 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:38.705378 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:38.705460 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:38.708024 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:38.708026 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:38.708026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:38.708035 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:38.710592 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.711081 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.711082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.711083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.711156 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:38.711402 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:38.711403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:38.711403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:38.711561 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:38.711638 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:38.711824 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 07:04:38.711914 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260808 07:04:38.714591 28038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e: Bootstrap starting.
I20260808 07:04:38.715533 28038 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:38.715866 28038 log.cc:824] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:38.717374 28038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e: No bootstrap required, opened a new log
I20260808 07:04:38.719704 28038 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b85f1f8f6241d5921288c2b423046e" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39713 } } peers { permanent_uuid: "cbfcca3fc2f147759da30e04c54309c2" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } } peers { permanent_uuid: "bcc13d5c4f584051b8f76106c53164ba" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } }
I20260808 07:04:38.719967 28038 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:38.720074 28038 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9b85f1f8f6241d5921288c2b423046e, State: Initialized, Role: FOLLOWER
I20260808 07:04:38.720280 28038 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b85f1f8f6241d5921288c2b423046e" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39713 } } peers { permanent_uuid: "cbfcca3fc2f147759da30e04c54309c2" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } } peers { permanent_uuid: "bcc13d5c4f584051b8f76106c53164ba" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } }
I20260808 07:04:38.720747 28038 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:38.720719 28240 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b85f1f8f6241d5921288c2b423046e" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39713 } } peers { permanent_uuid: "cbfcca3fc2f147759da30e04c54309c2" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } } peers { permanent_uuid: "bcc13d5c4f584051b8f76106c53164ba" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } } }
I20260808 07:04:38.720901 28240 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:04:38.723009 28253 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:04:38.723177 28253 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:04:38.726963 27962 master_runner.cc:263] No leader master found from master d9b85f1f8f6241d5921288c2b423046e
I20260808 07:04:38.727041 27962 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 07:04:38.752817 27985 tablet.cc:2369] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:04:38.873943 28149 tablet.cc:2369] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:04:38.922206 28240 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:04:38.922343 28240 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b85f1f8f6241d5921288c2b423046e" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39713 } } peers { permanent_uuid: "cbfcca3fc2f147759da30e04c54309c2" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } } peers { permanent_uuid: "bcc13d5c4f584051b8f76106c53164ba" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } }
I20260808 07:04:38.923049 28240 leader_election.cc:288] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbfcca3fc2f147759da30e04c54309c2 (127.13.207.61:44707), bcc13d5c4f584051b8f76106c53164ba (127.13.207.60:44011)
I20260808 07:04:38.923399 28091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9b85f1f8f6241d5921288c2b423046e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbfcca3fc2f147759da30e04c54309c2" is_pre_election: true
I20260808 07:04:38.923461 28172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9b85f1f8f6241d5921288c2b423046e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcc13d5c4f584051b8f76106c53164ba" is_pre_election: true
I20260808 07:04:38.923640 28091 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9b85f1f8f6241d5921288c2b423046e in term 0.
I20260808 07:04:38.923640 28172 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9b85f1f8f6241d5921288c2b423046e in term 0.
I20260808 07:04:38.923885 27982 leader_election.cc:302] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbfcca3fc2f147759da30e04c54309c2, d9b85f1f8f6241d5921288c2b423046e; no voters:
I20260808 07:04:38.924104 28240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:04:38.924167 28240 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:04:38.924197 28240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:38.925076 28240 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b85f1f8f6241d5921288c2b423046e" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39713 } } peers { permanent_uuid: "cbfcca3fc2f147759da30e04c54309c2" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } } peers { permanent_uuid: "bcc13d5c4f584051b8f76106c53164ba" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } }
I20260808 07:04:38.925256 28240 leader_election.cc:288] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [CANDIDATE]: Term 1 election: Requested vote from peers cbfcca3fc2f147759da30e04c54309c2 (127.13.207.61:44707), bcc13d5c4f584051b8f76106c53164ba (127.13.207.60:44011)
I20260808 07:04:38.925472 28091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9b85f1f8f6241d5921288c2b423046e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbfcca3fc2f147759da30e04c54309c2"
I20260808 07:04:38.925582 28091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:38.925792 28172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9b85f1f8f6241d5921288c2b423046e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcc13d5c4f584051b8f76106c53164ba"
I20260808 07:04:38.925984 28172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:38.926519 28091 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9b85f1f8f6241d5921288c2b423046e in term 1.
I20260808 07:04:38.926736 27982 leader_election.cc:302] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbfcca3fc2f147759da30e04c54309c2, d9b85f1f8f6241d5921288c2b423046e; no voters:
I20260808 07:04:38.926836 28240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:38.927003 28240 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [term 1 LEADER]: Becoming Leader. State: Replica: d9b85f1f8f6241d5921288c2b423046e, State: Running, Role: LEADER
I20260808 07:04:38.927114 28240 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b85f1f8f6241d5921288c2b423046e" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39713 } } peers { permanent_uuid: "cbfcca3fc2f147759da30e04c54309c2" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } } peers { permanent_uuid: "bcc13d5c4f584051b8f76106c53164ba" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } }
I20260808 07:04:38.927569 28172 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9b85f1f8f6241d5921288c2b423046e in term 1.
I20260808 07:04:38.927613 28255 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9b85f1f8f6241d5921288c2b423046e. Latest consensus state: current_term: 1 leader_uuid: "d9b85f1f8f6241d5921288c2b423046e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b85f1f8f6241d5921288c2b423046e" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39713 } } peers { permanent_uuid: "cbfcca3fc2f147759da30e04c54309c2" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } } peers { permanent_uuid: "bcc13d5c4f584051b8f76106c53164ba" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } } }
I20260808 07:04:38.927711 28255 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:38.927924 28257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:38.928138 28257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260808 07:04:38.928301 28069 tablet.cc:2369] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:04:38.929625 28091 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2 [term 1 FOLLOWER]: Refusing update from remote peer d9b85f1f8f6241d5921288c2b423046e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:38.929638 28172 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba [term 1 FOLLOWER]: Refusing update from remote peer d9b85f1f8f6241d5921288c2b423046e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:38.929997 28240 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbfcca3fc2f147759da30e04c54309c2" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:04:38.930135 28255 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcc13d5c4f584051b8f76106c53164ba" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:04:38.931456 28225 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9b85f1f8f6241d5921288c2b423046e. Latest consensus state: current_term: 1 leader_uuid: "d9b85f1f8f6241d5921288c2b423046e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b85f1f8f6241d5921288c2b423046e" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39713 } } peers { permanent_uuid: "cbfcca3fc2f147759da30e04c54309c2" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } } peers { permanent_uuid: "bcc13d5c4f584051b8f76106c53164ba" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } } }
I20260808 07:04:38.931594 28225 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:04:38.932301 28204 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9b85f1f8f6241d5921288c2b423046e. Latest consensus state: current_term: 1 leader_uuid: "d9b85f1f8f6241d5921288c2b423046e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b85f1f8f6241d5921288c2b423046e" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39713 } } peers { permanent_uuid: "cbfcca3fc2f147759da30e04c54309c2" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } } peers { permanent_uuid: "bcc13d5c4f584051b8f76106c53164ba" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } } }
I20260808 07:04:38.932400 28204 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:04:38.932991 28255 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d9b85f1f8f6241d5921288c2b423046e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b85f1f8f6241d5921288c2b423046e" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39713 } } peers { permanent_uuid: "cbfcca3fc2f147759da30e04c54309c2" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } } peers { permanent_uuid: "bcc13d5c4f584051b8f76106c53164ba" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } } }
I20260808 07:04:38.933096 28255 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:38.933147 28258 mvcc.cc:205] Tried to move back new op lower bound from 7316163292893401088 to 7316163292886102016. Current Snapshot: MvccSnapshot[applied={T|T < 7316163292893401088}]
I20260808 07:04:38.933358 28255 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d9b85f1f8f6241d5921288c2b423046e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b85f1f8f6241d5921288c2b423046e" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39713 } } peers { permanent_uuid: "cbfcca3fc2f147759da30e04c54309c2" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } } peers { permanent_uuid: "bcc13d5c4f584051b8f76106c53164ba" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } } }
I20260808 07:04:38.933461 28255 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:38.933658 28204 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d9b85f1f8f6241d5921288c2b423046e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b85f1f8f6241d5921288c2b423046e" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39713 } } peers { permanent_uuid: "cbfcca3fc2f147759da30e04c54309c2" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } } peers { permanent_uuid: "bcc13d5c4f584051b8f76106c53164ba" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } } }
I20260808 07:04:38.933707 28257 catalog_manager.cc:1347] Generated new cluster ID: b62b35b9d5c84519aedc20b8ad9554b7
I20260808 07:04:38.933743 28204 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:04:38.933754 28257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:38.934293 28225 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d9b85f1f8f6241d5921288c2b423046e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b85f1f8f6241d5921288c2b423046e" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 39713 } } peers { permanent_uuid: "cbfcca3fc2f147759da30e04c54309c2" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 44707 } } peers { permanent_uuid: "bcc13d5c4f584051b8f76106c53164ba" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 44011 } } }
I20260808 07:04:38.934396 28225 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:04:38.934537 28259 mvcc.cc:205] Tried to move back new op lower bound from 7316163292893401088 to 7316163292886102016. Current Snapshot: MvccSnapshot[applied={T|T < 7316163292893401088 or (T in {7316163292893401088})}]
I20260808 07:04:38.942664 28257 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:38.943466 28257 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:38.955185 28257 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9b85f1f8f6241d5921288c2b423046e: Generated new TSK 0
I20260808 07:04:38.955637 28257 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:04:39.676858 28215 catalog_manager.cc:1259] Loaded cluster ID: b62b35b9d5c84519aedc20b8ad9554b7
I20260808 07:04:39.676962 28215 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba: loading cluster ID for follower catalog manager: success
I20260808 07:04:39.677824 28215 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba: acquiring CA information for follower catalog manager: success
I20260808 07:04:39.678181 28215 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bcc13d5c4f584051b8f76106c53164ba: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:04:39.697156 28237 catalog_manager.cc:1259] Loaded cluster ID: b62b35b9d5c84519aedc20b8ad9554b7
I20260808 07:04:39.697245 28237 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2: loading cluster ID for follower catalog manager: success
I20260808 07:04:39.698177 28237 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2: acquiring CA information for follower catalog manager: success
I20260808 07:04:39.698521 28237 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cbfcca3fc2f147759da30e04c54309c2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:04:40.025342 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 27962
I20260808 07:04:40.032918 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 28046
I20260808 07:04:40.040378 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 28127
2026-08-08T07:04:40Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3217 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:40 dist-test-slave-8tdl krb5kdc[28272](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:40 dist-test-slave-8tdl krb5kdc[28272](info): set up 2 sockets
Aug 08 07:04:40 dist-test-slave-8tdl krb5kdc[28272](info): commencing operation
krb5kdc: starting...
W20260808 07:04:41.193440 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:04:41 dist-test-slave-8tdl krb5kdc[28272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:04:41.342955 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:46749
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:46585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:46749
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:41.432767 28286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:41.433068 28286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:41.433112 28286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:41.434423 28286 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:41.434506 28286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:41.434533 28286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:41.434566 28286 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:41.434609 28286 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:41.436308 28286 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:46585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:46749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:46749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:41.436623 28286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:41.436887 28286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:41.442113 28295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:41.442175 28293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:41.442127 28292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:41.442548 28286 server_base.cc:1034] running on GCE node
I20260808 07:04:41.442849 28286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:41.443149 28286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:41.444307 28286 hybrid_clock.cc:648] HybridClock initialized: now 1786172681444290 us; error 31 us; skew 500 ppm
Aug 08 07:04:41 dist-test-slave-8tdl krb5kdc[28272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172681, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:41.447218 28286 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:04:41.447727 28286 webserver.cc:460] Webserver started at http://127.13.207.62:44667/ using document root <none> and password file <none>
I20260808 07:04:41.447926 28286 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:41.447969 28286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:41.448071 28286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:41.449357 28286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "96ce6a30e4714d38939762ec76ef2435"
format_stamp: "Formatted at 2026-08-08 07:04:41 on dist-test-slave-8tdl"
server_key: "c3643b6193be355ee0eb11367b762419"
server_key_iv: "f6cbe07bde21eed0355b07330bb3f1f4"
server_key_version: "encryptionkey@0"
I20260808 07:04:41.449753 28286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "96ce6a30e4714d38939762ec76ef2435"
format_stamp: "Formatted at 2026-08-08 07:04:41 on dist-test-slave-8tdl"
server_key: "c3643b6193be355ee0eb11367b762419"
server_key_iv: "f6cbe07bde21eed0355b07330bb3f1f4"
server_key_version: "encryptionkey@0"
I20260808 07:04:41.451411 28286 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:41.452260 28302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:41.452503 28286 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:41.452632 28286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "96ce6a30e4714d38939762ec76ef2435"
format_stamp: "Formatted at 2026-08-08 07:04:41 on dist-test-slave-8tdl"
server_key: "c3643b6193be355ee0eb11367b762419"
server_key_iv: "f6cbe07bde21eed0355b07330bb3f1f4"
server_key_version: "encryptionkey@0"
I20260808 07:04:41.452715 28286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:41.481869 28286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:41.484534 28286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:41.484665 28286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:41.489061 28286 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:46749
I20260808 07:04:41.489151 28358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:46749 every 8 connection(s)
I20260808 07:04:41.489516 28286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:41.490515 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 28286
I20260808 07:04:41.490639 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:04:41.490794 28359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:41.490875 14140 external_mini_cluster.cc:1428] Setting key e94e114bb9941f74cac13b1c515c0e33
I20260808 07:04:41.495000 28359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96ce6a30e4714d38939762ec76ef2435: Bootstrap starting.
I20260808 07:04:41.495791 28359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96ce6a30e4714d38939762ec76ef2435: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:41.496186 28359 log.cc:824] T 00000000000000000000000000000000 P 96ce6a30e4714d38939762ec76ef2435: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:41.497041 28359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96ce6a30e4714d38939762ec76ef2435: No bootstrap required, opened a new log
Aug 08 07:04:41 dist-test-slave-8tdl krb5kdc[28272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:04:41.499678 28359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96ce6a30e4714d38939762ec76ef2435 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96ce6a30e4714d38939762ec76ef2435" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 46749 } }
I20260808 07:04:41.499904 28359 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96ce6a30e4714d38939762ec76ef2435 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:41.499956 28359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96ce6a30e4714d38939762ec76ef2435 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96ce6a30e4714d38939762ec76ef2435, State: Initialized, Role: FOLLOWER
I20260808 07:04:41.500149 28359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96ce6a30e4714d38939762ec76ef2435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96ce6a30e4714d38939762ec76ef2435" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 46749 } }
I20260808 07:04:41.500247 28359 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96ce6a30e4714d38939762ec76ef2435 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:41.500273 28359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96ce6a30e4714d38939762ec76ef2435 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:41.500360 28359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96ce6a30e4714d38939762ec76ef2435 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:41.501232 28359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96ce6a30e4714d38939762ec76ef2435 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96ce6a30e4714d38939762ec76ef2435" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 46749 } }
I20260808 07:04:41.501391 28359 leader_election.cc:302] T 00000000000000000000000000000000 P 96ce6a30e4714d38939762ec76ef2435 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96ce6a30e4714d38939762ec76ef2435; no voters:
I20260808 07:04:41.501658 28359 leader_election.cc:288] T 00000000000000000000000000000000 P 96ce6a30e4714d38939762ec76ef2435 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:41.501696 28364 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96ce6a30e4714d38939762ec76ef2435 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:41.501850 28364 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96ce6a30e4714d38939762ec76ef2435 [term 1 LEADER]: Becoming Leader. State: Replica: 96ce6a30e4714d38939762ec76ef2435, State: Running, Role: LEADER
I20260808 07:04:41.501998 28359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96ce6a30e4714d38939762ec76ef2435 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:41.502048 28364 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96ce6a30e4714d38939762ec76ef2435 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96ce6a30e4714d38939762ec76ef2435" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 46749 } }
I20260808 07:04:41.502583 28366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96ce6a30e4714d38939762ec76ef2435 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96ce6a30e4714d38939762ec76ef2435. Latest consensus state: current_term: 1 leader_uuid: "96ce6a30e4714d38939762ec76ef2435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96ce6a30e4714d38939762ec76ef2435" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 46749 } } }
I20260808 07:04:41.502826 28366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96ce6a30e4714d38939762ec76ef2435 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:41.502739 28365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96ce6a30e4714d38939762ec76ef2435 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96ce6a30e4714d38939762ec76ef2435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96ce6a30e4714d38939762ec76ef2435" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 46749 } } }
I20260808 07:04:41.503019 28365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96ce6a30e4714d38939762ec76ef2435 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:41.503077 28375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:41.503732 28375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:41.504346 28362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:41.491469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58904 (local address 127.13.207.62:46749)
0808 07:04:41.491720 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:41.491731 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:41.491770 (+ 39us) server_negotiation.cc:407] Connection header received
0808 07:04:41.492280 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:41.492289 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:41.492369 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:41.492543 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:41.493170 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:41.494745 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:41.495638 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:41.496366 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:41.496507 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:41.499360 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:41.499368 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:41.499373 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:41.499385 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:41.501880 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:41.502390 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:41.502393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:41.502393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:41.502521 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:41.502859 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:41.502860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:41.502860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:41.503233 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:41.503864 (+ 631us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:41.503966 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 07:04:41.504134 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":121,"threads_started":1}
I20260808 07:04:41.505429 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:04:41.506464 28375 catalog_manager.cc:1347] Generated new cluster ID: b2ef81bd06044090aa1dc7501013f6c2
I20260808 07:04:41.506533 28375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:41.519130 28375 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:41.519769 28375 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:41.527288 28375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96ce6a30e4714d38939762ec76ef2435: Generated new TSK 0
I20260808 07:04:41.527603 28375 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:04:41.560170 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 28286
2026-08-08T07:04:41Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1515 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:04:41 dist-test-slave-8tdl krb5kdc[28389](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:04:41 dist-test-slave-8tdl krb5kdc[28389](info): set up 2 sockets
Aug 08 07:04:41 dist-test-slave-8tdl krb5kdc[28389](info): commencing operation
krb5kdc: starting...
W20260808 07:04:42.679147 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:04:42 dist-test-slave-8tdl krb5kdc[28389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:04:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:04:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:04:42.839612 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:35665
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:35665
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:42.933279 28405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:42.933575 28405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:42.933606 28405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:42.935127 28405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:42.935186 28405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:42.935204 28405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:42.935217 28405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:42.935231 28405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:42.936992 28405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:35665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:35665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:42.937258 28405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:42.937469 28405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:42.943061 28413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:42.943378 28411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:42.943378 28410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:42.943643 28405 server_base.cc:1034] running on GCE node
I20260808 07:04:42.943953 28405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:42.944253 28405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:42.945443 28405 hybrid_clock.cc:648] HybridClock initialized: now 1786172682945429 us; error 49 us; skew 500 ppm
Aug 08 07:04:42 dist-test-slave-8tdl krb5kdc[28389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172682, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:42.954243 28405 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:04:42.954869 28405 webserver.cc:460] Webserver started at http://127.13.207.62:44923/ using document root <none> and password file <none>
I20260808 07:04:42.955114 28405 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:42.955166 28405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:42.955276 28405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:42.956571 28405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "a389cf61f07d4008917e3985b6386ea9"
format_stamp: "Formatted at 2026-08-08 07:04:42 on dist-test-slave-8tdl"
server_key: "28a0ab6191de97b30b83102bcdf528ce"
server_key_iv: "b80c6a9c3153b5fd9527c25aecac81c7"
server_key_version: "encryptionkey@0"
I20260808 07:04:42.956950 28405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "a389cf61f07d4008917e3985b6386ea9"
format_stamp: "Formatted at 2026-08-08 07:04:42 on dist-test-slave-8tdl"
server_key: "28a0ab6191de97b30b83102bcdf528ce"
server_key_iv: "b80c6a9c3153b5fd9527c25aecac81c7"
server_key_version: "encryptionkey@0"
I20260808 07:04:42.958539 28405 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:42.959597 28420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:42.959837 28405 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:42.959968 28405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "a389cf61f07d4008917e3985b6386ea9"
format_stamp: "Formatted at 2026-08-08 07:04:42 on dist-test-slave-8tdl"
server_key: "28a0ab6191de97b30b83102bcdf528ce"
server_key_iv: "b80c6a9c3153b5fd9527c25aecac81c7"
server_key_version: "encryptionkey@0"
I20260808 07:04:42.960060 28405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:42.977134 28405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:42.979795 28405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:42.979997 28405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:42.984700 28405 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:35665
I20260808 07:04:42.984783 28474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:35665 every 8 connection(s)
I20260808 07:04:42.985188 28405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:42.986452 28475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:42.986783 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 28405
I20260808 07:04:42.986925 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:04:42.987185 14140 external_mini_cluster.cc:1428] Setting key 028a814bbbf4bd9921a93a01e7df02e4
I20260808 07:04:42.990655 28475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9: Bootstrap starting.
I20260808 07:04:42.991510 28475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:42.991839 28475 log.cc:824] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:42.992714 28475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9: No bootstrap required, opened a new log
Aug 08 07:04:42 dist-test-slave-8tdl krb5kdc[28389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:04:42.995865 28475 raft_consensus.cc:348] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a389cf61f07d4008917e3985b6386ea9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35665 } }
I20260808 07:04:42.996054 28475 raft_consensus.cc:374] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:42.996114 28475 raft_consensus.cc:729] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a389cf61f07d4008917e3985b6386ea9, State: Initialized, Role: FOLLOWER
I20260808 07:04:42.996304 28475 consensus_queue.cc:260] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a389cf61f07d4008917e3985b6386ea9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35665 } }
I20260808 07:04:42.996428 28475 raft_consensus.cc:388] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:42.996482 28475 raft_consensus.cc:482] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:42.996546 28475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:42.997268 28475 raft_consensus.cc:504] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a389cf61f07d4008917e3985b6386ea9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35665 } }
I20260808 07:04:42.997396 28475 leader_election.cc:302] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a389cf61f07d4008917e3985b6386ea9; no voters:
I20260808 07:04:42.997608 28475 leader_election.cc:288] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:04:42.997670 28480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:42.997840 28480 raft_consensus.cc:686] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 1 LEADER]: Becoming Leader. State: Replica: a389cf61f07d4008917e3985b6386ea9, State: Running, Role: LEADER
I20260808 07:04:42.997931 28475 sys_catalog.cc:564] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:42.998003 28480 consensus_queue.cc:237] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a389cf61f07d4008917e3985b6386ea9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35665 } }
I20260808 07:04:42.998481 28480 sys_catalog.cc:455] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a389cf61f07d4008917e3985b6386ea9. Latest consensus state: current_term: 1 leader_uuid: "a389cf61f07d4008917e3985b6386ea9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a389cf61f07d4008917e3985b6386ea9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35665 } } }
I20260808 07:04:42.998761 28480 sys_catalog.cc:458] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:42.998661 28482 sys_catalog.cc:455] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a389cf61f07d4008917e3985b6386ea9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a389cf61f07d4008917e3985b6386ea9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35665 } } }
I20260808 07:04:42.999037 28482 sys_catalog.cc:458] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:42.999464 28491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:43.000097 28491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:43.000363 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:42.987795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39250 (local address 127.13.207.62:35665)
0808 07:04:42.988017 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:42.988029 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:42.988074 (+ 45us) server_negotiation.cc:407] Connection header received
0808 07:04:42.988623 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:42.988633 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:42.988707 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:42.988897 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:42.989596 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:42.991213 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:42.992207 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:42.992736 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:42.992901 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:42.995889 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:42.995899 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:42.995899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:42.995912 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:42.998050 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:42.998506 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:42.998510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:42.998511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:42.998622 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:42.998904 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:42.998905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:42.998905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:42.999279 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:42.999457 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:42.999944 (+ 487us) server_negotiation.cc:299] Negotiation successful
0808 07:04:43.000113 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":94,"threads_started":1}
I20260808 07:04:43.001852 28491 catalog_manager.cc:1347] Generated new cluster ID: 5abfafd5a61042a8bde82ec672ffe015
I20260808 07:04:43.001917 28491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:43.019447 28491 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:04:43.020278 28491 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:43.032011 28491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9: Generated new TSK 0
I20260808 07:04:43.032294 28491 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
I20260808 07:04:43.072000 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35665
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:43.169991 28503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:43.170290 28503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:43.170337 28503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:43.171898 28503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:43.172087 28503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:43.172149 28503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:04:43.174010 28503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:43.174356 28503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:43.174633 28503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:43.180545 28509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:43.180768 28511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:43.180970 28503 server_base.cc:1034] running on GCE node
W20260808 07:04:43.180545 28508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:43.181294 28503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:43.181595 28503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:43.182796 28503 hybrid_clock.cc:648] HybridClock initialized: now 1786172683182751 us; error 42 us; skew 500 ppm
Aug 08 07:04:43 dist-test-slave-8tdl krb5kdc[28389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172683, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:43.185928 28503 init.cc:377] Logged in from keytab as kudu/127.13.207.1@KRBTEST.COM (short username kudu)
I20260808 07:04:43.186488 28503 webserver.cc:460] Webserver started at http://127.13.207.1:39351/ using document root <none> and password file <none>
I20260808 07:04:43.186793 28503 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:43.186868 28503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:43.187022 28503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:43.188292 28503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0"
format_stamp: "Formatted at 2026-08-08 07:04:43 on dist-test-slave-8tdl"
server_key: "ab1bf00a476e707e96a93a5fe312c78d"
server_key_iv: "02c675c2c74e3b23a69e137807fa65eb"
server_key_version: "encryptionkey@0"
I20260808 07:04:43.188726 28503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0"
format_stamp: "Formatted at 2026-08-08 07:04:43 on dist-test-slave-8tdl"
server_key: "ab1bf00a476e707e96a93a5fe312c78d"
server_key_iv: "02c675c2c74e3b23a69e137807fa65eb"
server_key_version: "encryptionkey@0"
I20260808 07:04:43.190335 28503 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:04:43.191329 28519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:43.191633 28503 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:04:43.191776 28503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0"
format_stamp: "Formatted at 2026-08-08 07:04:43 on dist-test-slave-8tdl"
server_key: "ab1bf00a476e707e96a93a5fe312c78d"
server_key_iv: "02c675c2c74e3b23a69e137807fa65eb"
server_key_version: "encryptionkey@0"
I20260808 07:04:43.191867 28503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:43.204304 28503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:43.207345 28503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:43.207597 28503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:43.207886 28503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:43.208271 28503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:43.208338 28503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:43.208396 28503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:43.208425 28503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:43.215541 28503 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:46609
I20260808 07:04:43.215638 28637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:46609 every 8 connection(s)
I20260808 07:04:43.216013 28503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:04:43.219365 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 28503
I20260808 07:04:43.219508 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:04:43.219724 14140 external_mini_cluster.cc:1428] Setting key 8131da206d445a54bc831075c938eda7
Aug 08 07:04:43 dist-test-slave-8tdl krb5kdc[28389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172683, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:04:43.228344 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:43.216957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:33771 (local address 127.13.207.62:35665)
0808 07:04:43.217030 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:43.217032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:43.217264 (+ 232us) server_negotiation.cc:407] Connection header received
0808 07:04:43.217803 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:43.217805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:43.217818 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:43.217900 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:43.219307 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.220016 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.221493 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.221735 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.221806 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:43.224603 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:43.224605 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:43.224605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:43.224625 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:43.226378 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:43.227033 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:43.227034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:43.227034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:43.227103 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:43.227532 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:43.227533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:43.227534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:43.227669 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:43.227730 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:43.228151 (+ 421us) server_negotiation.cc:299] Negotiation successful
0808 07:04:43.228257 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:04:43.228569 28640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:43.216865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.1:33771)
0808 07:04:43.217080 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:43.217126 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:43.217672 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:43.218174 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:43.218178 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:43.218915 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:43.219162 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:43.219171 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.220131 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.220132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:43.221373 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:43.221377 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.221834 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.221835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:43.222003 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:43.222780 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:43.222791 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:43.224474 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:43.226480 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:43.226492 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:43.226495 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:43.226926 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:43.227193 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:43.227194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:43.227194 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:43.227363 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:43.227746 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:43.227755 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:43.227954 (+ 199us) client_negotiation.cc:769] Sending connection context
0808 07:04:43.228078 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 07:04:43.228289 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":105,"threads_started":1}
I20260808 07:04:43.229219 28638 heartbeater.cc:347] Connected to a master server at 127.13.207.62:35665
I20260808 07:04:43.229422 28638 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:43.229672 28638 heartbeater.cc:511] Master 127.13.207.62:35665 requested a full tablet report, sending...
I20260808 07:04:43.230386 28438 ts_manager.cc:194] Registered new tserver with Master: c5bf2d6c97e14ac5b51efbcc4e118fa0 (127.13.207.1:46609)
WARNING: no policy specified for kudu/127.13.207.2@KRBTEST.COM; defaulting to no policy
I20260808 07:04:43.231184 28438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:33771
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
I20260808 07:04:43.259260 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35665
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:43.351783 28644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:43.352101 28644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:43.352154 28644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:43.354040 28644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:43.354132 28644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:43.354182 28644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:04:43.356052 28644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:43.356388 28644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:43.356654 28644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:43.362401 28649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:43.362396 28652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:43.362390 28650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:43.362644 28644 server_base.cc:1034] running on GCE node
I20260808 07:04:43.362987 28644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:43.363301 28644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:43.364470 28644 hybrid_clock.cc:648] HybridClock initialized: now 1786172683364439 us; error 48 us; skew 500 ppm
Aug 08 07:04:43 dist-test-slave-8tdl krb5kdc[28389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172683, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:43.367477 28644 init.cc:377] Logged in from keytab as kudu/127.13.207.2@KRBTEST.COM (short username kudu)
I20260808 07:04:43.367919 28644 webserver.cc:460] Webserver started at http://127.13.207.2:37257/ using document root <none> and password file <none>
I20260808 07:04:43.368167 28644 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:43.368251 28644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:43.368392 28644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:43.369603 28644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "b33e8b828e6a42ec9eb33f11aeb13838"
format_stamp: "Formatted at 2026-08-08 07:04:43 on dist-test-slave-8tdl"
server_key: "e2d53efeadb45a7a20a4488782367aea"
server_key_iv: "b626f2ae970fc0293e85e4d2cf190a83"
server_key_version: "encryptionkey@0"
I20260808 07:04:43.370000 28644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "b33e8b828e6a42ec9eb33f11aeb13838"
format_stamp: "Formatted at 2026-08-08 07:04:43 on dist-test-slave-8tdl"
server_key: "e2d53efeadb45a7a20a4488782367aea"
server_key_iv: "b626f2ae970fc0293e85e4d2cf190a83"
server_key_version: "encryptionkey@0"
I20260808 07:04:43.371688 28644 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:04:43.372648 28660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:43.373039 28644 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:43.373136 28644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "b33e8b828e6a42ec9eb33f11aeb13838"
format_stamp: "Formatted at 2026-08-08 07:04:43 on dist-test-slave-8tdl"
server_key: "e2d53efeadb45a7a20a4488782367aea"
server_key_iv: "b626f2ae970fc0293e85e4d2cf190a83"
server_key_version: "encryptionkey@0"
I20260808 07:04:43.373255 28644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:43.387750 28644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:43.390837 28644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:43.391150 28644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:43.391481 28644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:43.391851 28644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:43.391886 28644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:43.391911 28644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:43.391923 28644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:43.399631 28644 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:40977
I20260808 07:04:43.399717 28786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:40977 every 8 connection(s)
I20260808 07:04:43.400097 28644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:04:43.406500 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 28644
I20260808 07:04:43.406641 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:04:43.407037 14140 external_mini_cluster.cc:1428] Setting key c8ff14d4879e70500a8e62ada81c50c0
Aug 08 07:04:43 dist-test-slave-8tdl krb5kdc[28389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172683, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:04:43.414286 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:43.400989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:41529 (local address 127.13.207.62:35665)
0808 07:04:43.401225 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:43.401227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:43.401460 (+ 233us) server_negotiation.cc:407] Connection header received
0808 07:04:43.402785 (+ 1325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:43.402787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:43.402805 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:43.402955 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:43.404084 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.404680 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.405829 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.406052 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.406282 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:43.409762 (+ 3480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:43.409765 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:43.409765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:43.409773 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:43.411867 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:43.412569 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:43.412571 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:43.412571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:43.412646 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:43.413151 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:43.413152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:43.413152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:43.413375 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:43.413453 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:43.414015 (+ 562us) server_negotiation.cc:299] Negotiation successful
0808 07:04:43.414213 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203}
I20260808 07:04:43.414443 28789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:43.400959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.2:41529)
0808 07:04:43.401232 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:43.401289 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:43.402346 (+ 1057us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:43.403083 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:43.403093 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:43.403738 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:43.403961 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:43.403966 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.404794 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.404795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:43.405713 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:43.405716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.406356 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.406358 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:43.406597 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:43.407490 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:43.407502 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:43.409617 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:43.411995 (+ 2378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:43.412011 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:43.412015 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:43.412422 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:43.412737 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:43.412739 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:43.412739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:43.413040 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:43.413465 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:43.413477 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:43.413704 (+ 227us) client_negotiation.cc:769] Sending connection context
0808 07:04:43.413853 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 07:04:43.414099 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":119,"threads_started":1}
I20260808 07:04:43.415282 28787 heartbeater.cc:347] Connected to a master server at 127.13.207.62:35665
I20260808 07:04:43.415412 28787 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:43.415704 28787 heartbeater.cc:511] Master 127.13.207.62:35665 requested a full tablet report, sending...
I20260808 07:04:43.416205 28438 ts_manager.cc:194] Registered new tserver with Master: b33e8b828e6a42ec9eb33f11aeb13838 (127.13.207.2:40977)
I20260808 07:04:43.416617 28438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:41529
WARNING: no policy specified for kudu/127.13.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
I20260808 07:04:43.443938 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35665
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:43.533730 28793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:43.534008 28793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:43.534053 28793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:43.535494 28793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:43.535583 28793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:43.535640 28793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:04:43.537755 28793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:43.538120 28793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:43.538395 28793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:43.544410 28799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:43.544454 28801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:43.544546 28798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:43.545338 28793 server_base.cc:1034] running on GCE node
I20260808 07:04:43.545656 28793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:43.545997 28793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:43.547178 28793 hybrid_clock.cc:648] HybridClock initialized: now 1786172683547093 us; error 127 us; skew 500 ppm
Aug 08 07:04:43 dist-test-slave-8tdl krb5kdc[28389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172683, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:43.550244 28793 init.cc:377] Logged in from keytab as kudu/127.13.207.3@KRBTEST.COM (short username kudu)
I20260808 07:04:43.550824 28793 webserver.cc:460] Webserver started at http://127.13.207.3:36187/ using document root <none> and password file <none>
I20260808 07:04:43.551100 28793 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:43.551149 28793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:43.551290 28793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:04:43.552493 28793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "f6399499f2a242bfb972f2e51cf00a55"
format_stamp: "Formatted at 2026-08-08 07:04:43 on dist-test-slave-8tdl"
server_key: "6a9d05aa93944fc2d02362d56cd89181"
server_key_iv: "75793b2ebb7238a191ca02914f98b7d3"
server_key_version: "encryptionkey@0"
I20260808 07:04:43.553263 28793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "f6399499f2a242bfb972f2e51cf00a55"
format_stamp: "Formatted at 2026-08-08 07:04:43 on dist-test-slave-8tdl"
server_key: "6a9d05aa93944fc2d02362d56cd89181"
server_key_iv: "75793b2ebb7238a191ca02914f98b7d3"
server_key_version: "encryptionkey@0"
I20260808 07:04:43.555339 28793 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.004s
I20260808 07:04:43.556631 28808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:43.557206 28793 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:43.557350 28793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "f6399499f2a242bfb972f2e51cf00a55"
format_stamp: "Formatted at 2026-08-08 07:04:43 on dist-test-slave-8tdl"
server_key: "6a9d05aa93944fc2d02362d56cd89181"
server_key_iv: "75793b2ebb7238a191ca02914f98b7d3"
server_key_version: "encryptionkey@0"
I20260808 07:04:43.557454 28793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:43.572322 28793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:43.575240 28793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:43.575436 28793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:43.575723 28793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:43.576069 28793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:43.576128 28793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:43.576174 28793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:43.576198 28793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:43.583101 28793 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:40299
I20260808 07:04:43.583156 28929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:40299 every 8 connection(s)
I20260808 07:04:43.583562 28793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:04:43.590942 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 28793
I20260808 07:04:43.591053 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:04:43.591264 14140 external_mini_cluster.cc:1428] Setting key 40b72f80b9be65e8fa0948ff46f2bbab
Aug 08 07:04:43 dist-test-slave-8tdl krb5kdc[28389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172683, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:04:43.595662 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:43.584541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:46757 (local address 127.13.207.62:35665)
0808 07:04:43.584634 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:43.584637 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:43.584900 (+ 263us) server_negotiation.cc:407] Connection header received
0808 07:04:43.585556 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:43.585557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:43.585572 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:43.585655 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:43.586970 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.587543 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.588718 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.588958 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.589051 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:43.592058 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:43.592059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:43.592059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:43.592078 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:43.593761 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:43.594276 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:43.594277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:43.594278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:43.594347 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:43.594887 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:43.594889 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:43.594889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:43.595073 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:43.595149 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:43.595495 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 07:04:43.595582 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 07:04:43.595979 28932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:43.584433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.3:46757)
0808 07:04:43.584686 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:43.584746 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:43.585424 (+ 678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:43.585739 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:43.585744 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:43.586489 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:43.586852 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:43.586858 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.587665 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.587666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:43.588601 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:43.588604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.589066 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.589068 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:43.589235 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:43.589861 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:43.589873 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:43.591928 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:43.593864 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:43.593874 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:43.593877 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:43.594169 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:43.594459 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:43.594460 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:43.594461 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:43.594607 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:43.595186 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:43.595195 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:43.595345 (+ 150us) client_negotiation.cc:769] Sending connection context
0808 07:04:43.595454 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 07:04:43.595663 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":116,"threads_started":1}
I20260808 07:04:43.596596 28930 heartbeater.cc:347] Connected to a master server at 127.13.207.62:35665
I20260808 07:04:43.596714 28930 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:43.596879 28930 heartbeater.cc:511] Master 127.13.207.62:35665 requested a full tablet report, sending...
I20260808 07:04:43.597293 28438 ts_manager.cc:194] Registered new tserver with Master: f6399499f2a242bfb972f2e51cf00a55 (127.13.207.3:40299)
I20260808 07:04:43.597671 28438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:46757
I20260808 07:04:43.603569 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:04:43.613788 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:43.605155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39262 (local address 127.13.207.62:35665)
0808 07:04:43.605282 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:43.605285 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:43.605296 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:04:43.605358 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:43.605359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:43.605370 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:43.605435 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:43.606115 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.606643 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.607603 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.607831 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.607917 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:43.609757 (+ 1840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:43.609759 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:43.609759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:43.609774 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:43.612010 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:43.612591 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:43.612593 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:43.612594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:43.612677 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:43.613080 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:43.613081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:43.613081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:43.613273 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:43.613399 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:43.613562 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 07:04:43.613675 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260808 07:04:43.616289 28438 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39262:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:04:43.618098 28438 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:04:43.625638 28947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:43.621757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53166 (local address 127.13.207.2:40977)
0808 07:04:43.621974 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:43.621979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:43.621988 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:04:43.622036 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:43.622038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:43.622053 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:43.622128 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:43.622319 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.623249 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.624554 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.625113 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.625211 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:43.625296 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:43.625487 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 07:04:43.625538 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":135,"threads_started":1}
I20260808 07:04:43.625624 28946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:43.621406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:40977 (local address 127.0.0.1:53166)
0808 07:04:43.621724 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:43.621737 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:43.621803 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:43.622163 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:43.622167 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:43.622169 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:43.622238 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:43.622240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.623368 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.623369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:43.624456 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:43.624459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.625224 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.625226 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:43.625331 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:43.625355 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 07:04:43.625463 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 07:04:43.625535 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":163,"threads_started":1}
I20260808 07:04:43.626562 28715 tablet_service.cc:1467] Processing CreateTablet for tablet 7c6b7eb400d5488281edd59397286e0f (DEFAULT_TABLE table=test-table [id=12be4d82a36442a9a74a7d2d2e9de485]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:43.626974 28942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:43.621190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:46609 (local address 127.0.0.1:39810)
0808 07:04:43.621623 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:43.621632 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:43.621707 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:43.622450 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:43.622451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:43.622454 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:43.622499 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:43.622500 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.623866 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.623866 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:43.625156 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:43.625159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.626587 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.626589 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:43.626658 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:43.626665 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 07:04:43.626750 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 07:04:43.626804 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":82,"spinlock_wait_cycles":384,"thread_start_us":110,"threads_started":1}
I20260808 07:04:43.627004 28945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:43.621455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39810 (local address 127.13.207.1:46609)
0808 07:04:43.621893 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:43.621898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:43.621911 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:04:43.621967 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:43.621970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:43.621988 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:43.622078 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:43.622576 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.623736 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.625269 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.626482 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.626576 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:43.626642 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:43.626782 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 07:04:43.626917 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":77,"threads_started":1}
I20260808 07:04:43.627148 28715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c6b7eb400d5488281edd59397286e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:43.627887 28571 tablet_service.cc:1467] Processing CreateTablet for tablet 7c6b7eb400d5488281edd59397286e0f (DEFAULT_TABLE table=test-table [id=12be4d82a36442a9a74a7d2d2e9de485]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:43.628237 28571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c6b7eb400d5488281edd59397286e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:43.630455 28948 tablet_bootstrap.cc:492] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838: Bootstrap starting.
I20260808 07:04:43.631148 28948 tablet_bootstrap.cc:654] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:43.631248 28949 tablet_bootstrap.cc:492] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0: Bootstrap starting.
I20260808 07:04:43.631697 28948 log.cc:824] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:43.631907 28944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:43.621328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44674 (local address 127.13.207.3:40299)
0808 07:04:43.622158 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:43.622169 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:43.622180 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:04:43.622223 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:43.622225 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:43.622241 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:43.622349 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:43.622489 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.623726 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.625271 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.627435 (+ 2164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.627533 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:43.627595 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:43.631715 (+ 4120us) server_negotiation.cc:299] Negotiation successful
0808 07:04:43.631785 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":164,"threads_started":1}
I20260808 07:04:43.631923 28949 tablet_bootstrap.cc:654] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:43.632239 28943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:43.621190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:40299 (local address 127.0.0.1:44674)
0808 07:04:43.621378 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:43.621424 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:43.621491 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:43.622366 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:43.622366 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:43.622368 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:43.622406 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:43.622406 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.623858 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.623859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:43.625157 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:43.625159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.631455 (+ 6296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.631456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:43.631559 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:43.631566 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 07:04:43.632055 (+ 489us) client_negotiation.cc:240] Negotiation successful
0808 07:04:43.632134 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":44,"thread_start_us":67,"threads_started":1}
I20260808 07:04:43.632517 28949 log.cc:824] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:43.632848 28948 tablet_bootstrap.cc:492] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838: No bootstrap required, opened a new log
I20260808 07:04:43.632925 28948 ts_tablet_manager.cc:1397] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:04:43.633055 28860 tablet_service.cc:1467] Processing CreateTablet for tablet 7c6b7eb400d5488281edd59397286e0f (DEFAULT_TABLE table=test-table [id=12be4d82a36442a9a74a7d2d2e9de485]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:43.633349 28860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c6b7eb400d5488281edd59397286e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:43.634545 28948 raft_consensus.cc:348] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } }
I20260808 07:04:43.634763 28948 raft_consensus.cc:374] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:43.634830 28948 raft_consensus.cc:729] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b33e8b828e6a42ec9eb33f11aeb13838, State: Initialized, Role: FOLLOWER
I20260808 07:04:43.634963 28948 consensus_queue.cc:260] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } }
I20260808 07:04:43.635345 28787 heartbeater.cc:503] Master 127.13.207.62:35665 was elected leader, sending a full tablet report...
I20260808 07:04:43.635638 28948 ts_tablet_manager.cc:1428] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:04:43.636435 28949 tablet_bootstrap.cc:492] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0: No bootstrap required, opened a new log
I20260808 07:04:43.636523 28949 ts_tablet_manager.cc:1397] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 07:04:43.636513 28953 tablet_bootstrap.cc:492] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55: Bootstrap starting.
I20260808 07:04:43.637270 28953 tablet_bootstrap.cc:654] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:43.637897 28953 log.cc:824] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:43.638799 28953 tablet_bootstrap.cc:492] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55: No bootstrap required, opened a new log
I20260808 07:04:43.638882 28953 ts_tablet_manager.cc:1397] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:04:43.639081 28949 raft_consensus.cc:348] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } }
I20260808 07:04:43.639278 28949 raft_consensus.cc:374] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:43.639317 28949 raft_consensus.cc:729] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bf2d6c97e14ac5b51efbcc4e118fa0, State: Initialized, Role: FOLLOWER
I20260808 07:04:43.639448 28949 consensus_queue.cc:260] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } }
I20260808 07:04:43.639758 28949 ts_tablet_manager.cc:1428] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:04:43.639781 28638 heartbeater.cc:503] Master 127.13.207.62:35665 was elected leader, sending a full tablet report...
I20260808 07:04:43.640655 28953 raft_consensus.cc:348] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } }
I20260808 07:04:43.640786 28953 raft_consensus.cc:374] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:43.640808 28953 raft_consensus.cc:729] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6399499f2a242bfb972f2e51cf00a55, State: Initialized, Role: FOLLOWER
I20260808 07:04:43.640952 28953 consensus_queue.cc:260] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } }
I20260808 07:04:43.641244 28953 ts_tablet_manager.cc:1428] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:04:43.641274 28930 heartbeater.cc:503] Master 127.13.207.62:35665 was elected leader, sending a full tablet report...
W20260808 07:04:43.650894 28788 tablet.cc:2369] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:04:43.716951 28639 tablet.cc:2369] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:04:43.823975 28956 raft_consensus.cc:482] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:04:43.824252 28956 raft_consensus.cc:504] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } }
I20260808 07:04:43.824749 28956 leader_election.cc:288] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5bf2d6c97e14ac5b51efbcc4e118fa0 (127.13.207.1:46609), b33e8b828e6a42ec9eb33f11aeb13838 (127.13.207.2:40977)
I20260808 07:04:43.827986 28947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:43.825084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:54075 (local address 127.13.207.2:40977)
0808 07:04:43.825137 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:43.825139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:43.825153 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:04:43.825265 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:43.825267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:43.825287 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:43.825357 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:43.825534 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.826056 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.827184 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.827596 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.827690 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:43.827700 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:43.827865 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 07:04:43.827909 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:04:43.827986 28932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:43.825007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:40977 (local address 127.13.207.3:54075)
0808 07:04:43.825079 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:43.825093 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:43.825150 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:43.825363 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:43.825365 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:43.825367 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:43.825455 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:43.825457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.826161 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.826162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:43.827078 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:43.827081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.827689 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.827690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:43.827779 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:43.827783 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:04:43.827855 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 07:04:43.827905 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":30}
I20260808 07:04:43.828145 28945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:43.825114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:52773 (local address 127.13.207.1:46609)
0808 07:04:43.825201 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:43.825203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:43.825386 (+ 183us) server_negotiation.cc:407] Connection header received
0808 07:04:43.825463 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:43.825464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:43.825476 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:43.825554 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:43.825722 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.826215 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.827341 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.827712 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.827818 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:43.827827 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:43.828013 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 07:04:43.828058 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 07:04:43.828337 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:43.825010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:46609 (local address 127.13.207.3:52773)
0808 07:04:43.825249 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:43.825262 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:43.825398 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:43.825560 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:43.825561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:43.825563 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:43.825648 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:43.825650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.826314 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.826315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:43.827222 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:43.827226 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.827819 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.827820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:43.827901 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:43.827911 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 07:04:43.827976 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 07:04:43.828026 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":64,"thread_start_us":101,"threads_started":1}
I20260808 07:04:43.828439 28737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c6b7eb400d5488281edd59397286e0f" candidate_uuid: "f6399499f2a242bfb972f2e51cf00a55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" is_pre_election: true
I20260808 07:04:43.828655 28737 raft_consensus.cc:2454] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6399499f2a242bfb972f2e51cf00a55 in term 0.
I20260808 07:04:43.828771 28591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c6b7eb400d5488281edd59397286e0f" candidate_uuid: "f6399499f2a242bfb972f2e51cf00a55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" is_pre_election: true
I20260808 07:04:43.828948 28591 raft_consensus.cc:2454] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6399499f2a242bfb972f2e51cf00a55 in term 0.
I20260808 07:04:43.828965 28809 leader_election.cc:302] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b33e8b828e6a42ec9eb33f11aeb13838, f6399499f2a242bfb972f2e51cf00a55; no voters:
I20260808 07:04:43.829118 28956 raft_consensus.cc:2781] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:04:43.829182 28956 raft_consensus.cc:482] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:04:43.829208 28956 raft_consensus.cc:3037] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:43.830049 28956 raft_consensus.cc:504] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } }
I20260808 07:04:43.830228 28956 leader_election.cc:288] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55 [CANDIDATE]: Term 1 election: Requested vote from peers c5bf2d6c97e14ac5b51efbcc4e118fa0 (127.13.207.1:46609), b33e8b828e6a42ec9eb33f11aeb13838 (127.13.207.2:40977)
I20260808 07:04:43.830435 28737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c6b7eb400d5488281edd59397286e0f" candidate_uuid: "f6399499f2a242bfb972f2e51cf00a55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b33e8b828e6a42ec9eb33f11aeb13838"
I20260808 07:04:43.830542 28737 raft_consensus.cc:3037] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:43.830585 28591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c6b7eb400d5488281edd59397286e0f" candidate_uuid: "f6399499f2a242bfb972f2e51cf00a55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0"
I20260808 07:04:43.830718 28591 raft_consensus.cc:3037] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:43.831457 28737 raft_consensus.cc:2454] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6399499f2a242bfb972f2e51cf00a55 in term 1.
I20260808 07:04:43.831650 28809 leader_election.cc:302] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b33e8b828e6a42ec9eb33f11aeb13838, f6399499f2a242bfb972f2e51cf00a55; no voters:
I20260808 07:04:43.831653 28591 raft_consensus.cc:2454] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6399499f2a242bfb972f2e51cf00a55 in term 1.
I20260808 07:04:43.831866 28956 raft_consensus.cc:2781] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:43.832034 28956 raft_consensus.cc:686] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55 [term 1 LEADER]: Becoming Leader. State: Replica: f6399499f2a242bfb972f2e51cf00a55, State: Running, Role: LEADER
I20260808 07:04:43.832155 28956 consensus_queue.cc:237] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } }
I20260808 07:04:43.833184 28438 catalog_manager.cc:5495] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6399499f2a242bfb972f2e51cf00a55 (127.13.207.3). New cstate: current_term: 1 leader_uuid: "f6399499f2a242bfb972f2e51cf00a55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } health_report { overall_health: UNKNOWN } } }
W20260808 07:04:43.834173 28931 tablet.cc:2369] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:04:43.856629 28944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:43.853347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44684 (local address 127.13.207.3:40299)
0808 07:04:43.853453 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:43.853455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:43.853465 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:04:43.853521 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:43.853522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:43.853578 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:43.853653 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:04:43.853863 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.854413 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.855489 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:43.855695 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:43.855765 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:43.856092 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:04:43.856220 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:04:43.856322 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:43.856417 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 07:04:43.856445 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 07:04:43.859009 28737 raft_consensus.cc:1261] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838 [term 1 FOLLOWER]: Refusing update from remote peer f6399499f2a242bfb972f2e51cf00a55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:43.859009 28591 raft_consensus.cc:1261] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 1 FOLLOWER]: Refusing update from remote peer f6399499f2a242bfb972f2e51cf00a55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:43.859449 28958 consensus_queue.cc:1035] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:04:43.859685 28956 consensus_queue.cc:1035] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:04:43.863126 28963 mvcc.cc:205] Tried to move back new op lower bound from 7316163313082564608 to 7316163312977125376. Current Snapshot: MvccSnapshot[applied={T|T < 7316163313082564608}]
I20260808 07:04:43.862882 28962 mvcc.cc:205] Tried to move back new op lower bound from 7316163313082564608 to 7316163312977125376. Current Snapshot: MvccSnapshot[applied={T|T < 7316163313082564608}]
I20260808 07:04:43.864598 28964 mvcc.cc:205] Tried to move back new op lower bound from 7316163313082564608 to 7316163312977125376. Current Snapshot: MvccSnapshot[applied={T|T < 7316163313082564608 or (T in {7316163313082564608})}]
I20260808 07:04:43.866333 28438 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:04:43.866513 28438 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:04:43.867497 28438 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9: Sending DeleteTablet for 3 replicas of tablet 7c6b7eb400d5488281edd59397286e0f
I20260808 07:04:43.867902 28860 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c6b7eb400d5488281edd59397286e0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:43 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:44674
I20260808 07:04:43.867913 28571 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c6b7eb400d5488281edd59397286e0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:43 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:39810
I20260808 07:04:43.868216 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 28503
I20260808 07:04:43.868270 28971 tablet_replica.cc:330] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55: stopping tablet replica
I20260808 07:04:43.868273 28715 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c6b7eb400d5488281edd59397286e0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:43 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:53166
I20260808 07:04:43.868386 28971 raft_consensus.cc:2229] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:04:43.868527 28971 raft_consensus.cc:2258] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:04:43.870816 28971 ts_tablet_manager.cc:1905] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:04:43.873327 28971 ts_tablet_manager.cc:1918] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:04:43.873463 28971 log.cc:1196] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/7c6b7eb400d5488281edd59397286e0f
I20260808 07:04:43.873847 28971 ts_tablet_manager.cc:1939] T 7c6b7eb400d5488281edd59397286e0f P f6399499f2a242bfb972f2e51cf00a55: Deleting consensus metadata
I20260808 07:04:43.874487 28421 catalog_manager.cc:4841] TS f6399499f2a242bfb972f2e51cf00a55 (127.13.207.3:40299): tablet 7c6b7eb400d5488281edd59397286e0f (table test-table [id=12be4d82a36442a9a74a7d2d2e9de485]) successfully deleted
I20260808 07:04:43.874882 28973 tablet_replica.cc:330] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838: stopping tablet replica
I20260808 07:04:43.875100 28973 raft_consensus.cc:2229] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:04:43.875291 28973 raft_consensus.cc:2258] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:04:43.875932 28973 ts_tablet_manager.cc:1905] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:04:43.878893 28973 ts_tablet_manager.cc:1918] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:04:43.879099 28973 log.cc:1196] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/wal/wals/7c6b7eb400d5488281edd59397286e0f
W20260808 07:04:43.879213 28422 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.207.1:46609 (error 108)
I20260808 07:04:43.879351 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 28644
I20260808 07:04:43.880391 28943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:43.879833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:46609 (local address 127.0.0.1:39816)
0808 07:04:43.880297 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:43.880345 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.1:46609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439}
W20260808 07:04:43.880704 28422 catalog_manager.cc:4584] TS c5bf2d6c97e14ac5b51efbcc4e118fa0 (127.13.207.1:46609): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7c6b7eb400d5488281edd59397286e0f: Network error: Client connection negotiation failed: client connection to 127.13.207.1:46609: connect: Connection refused (error 111)
I20260808 07:04:43.888515 28943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:43.888346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:40977 (local address 127.0.0.1:53174)
0808 07:04:43.888419 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:43.888461 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.2:40977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 07:04:43.891626 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 28793
I20260808 07:04:43.901835 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 28405
I20260808 07:04:43.910226 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:35665
--webserver_interface=127.13.207.62
--webserver_port=44923
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:35665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:43.999315 28975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:43.999557 28975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:43.999583 28975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:44.000907 28975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:44.001312 28975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:44.001353 28975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:44.001367 28975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:44.001381 28975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:44.003706 28975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:35665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:35665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=44923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:44.004091 28975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:44.004411 28975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:04:44.018998 28975 server_base.cc:1034] running on GCE node
W20260808 07:04:44.019402 28982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:44.018894 28981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:44.018913 28984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:44.019984 28975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:44.020360 28975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:44.021637 28975 hybrid_clock.cc:648] HybridClock initialized: now 1786172684021568 us; error 83 us; skew 500 ppm
Aug 08 07:04:44 dist-test-slave-8tdl krb5kdc[28389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172684, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:44.025770 28975 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:04:44.026371 28975 webserver.cc:460] Webserver started at http://127.13.207.62:44923/ using document root <none> and password file <none>
I20260808 07:04:44.026660 28975 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:44.026825 28975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:44.028594 28975 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:44.029613 28993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:44.030109 28975 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:44.030224 28975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "a389cf61f07d4008917e3985b6386ea9"
format_stamp: "Formatted at 2026-08-08 07:04:42 on dist-test-slave-8tdl"
server_key: "28a0ab6191de97b30b83102bcdf528ce"
server_key_iv: "b80c6a9c3153b5fd9527c25aecac81c7"
server_key_version: "encryptionkey@0"
I20260808 07:04:44.030853 28975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:44.057123 28975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:44.060189 28975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:44.060410 28975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:44.065357 28975 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:35665
I20260808 07:04:44.065426 29050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:35665 every 8 connection(s)
I20260808 07:04:44.065891 28975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:44.067477 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 28975
I20260808 07:04:44.067792 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:46609
--local_ip_for_outbound_sockets=127.13.207.1
--tserver_master_addrs=127.13.207.62:35665
--webserver_port=39351
--webserver_interface=127.13.207.1
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:04:44.068060 29051 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:04:44.068953 29051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9: Bootstrap starting.
I20260808 07:04:44.079337 29051 log.cc:824] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:44.081665 29051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:04:44.082036 29051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9: Bootstrap complete.
I20260808 07:04:44.084914 29051 raft_consensus.cc:348] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a389cf61f07d4008917e3985b6386ea9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35665 } }
I20260808 07:04:44.085263 29051 raft_consensus.cc:729] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a389cf61f07d4008917e3985b6386ea9, State: Initialized, Role: FOLLOWER
I20260808 07:04:44.085510 29051 consensus_queue.cc:260] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a389cf61f07d4008917e3985b6386ea9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35665 } }
I20260808 07:04:44.085615 29051 raft_consensus.cc:388] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:44.085681 29051 raft_consensus.cc:482] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:44.085734 29051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:04:44.087007 29051 raft_consensus.cc:504] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a389cf61f07d4008917e3985b6386ea9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35665 } }
I20260808 07:04:44.087144 29051 leader_election.cc:302] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a389cf61f07d4008917e3985b6386ea9; no voters:
I20260808 07:04:44.087352 29051 leader_election.cc:288] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 07:04:44.087433 29055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 07:04:44.087610 29055 raft_consensus.cc:686] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 2 LEADER]: Becoming Leader. State: Replica: a389cf61f07d4008917e3985b6386ea9, State: Running, Role: LEADER
I20260808 07:04:44.087663 29051 sys_catalog.cc:564] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:44.087726 29055 consensus_queue.cc:237] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a389cf61f07d4008917e3985b6386ea9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35665 } }
I20260808 07:04:44.088014 29055 sys_catalog.cc:455] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a389cf61f07d4008917e3985b6386ea9. Latest consensus state: current_term: 2 leader_uuid: "a389cf61f07d4008917e3985b6386ea9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a389cf61f07d4008917e3985b6386ea9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35665 } } }
I20260808 07:04:44.088106 29055 sys_catalog.cc:458] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:44.088016 29056 sys_catalog.cc:455] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a389cf61f07d4008917e3985b6386ea9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a389cf61f07d4008917e3985b6386ea9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35665 } } }
I20260808 07:04:44.088340 29056 sys_catalog.cc:458] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:44.088289 29060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:44.089375 29060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:44.089527 29060 catalog_manager.cc:1259] Loaded cluster ID: 5abfafd5a61042a8bde82ec672ffe015
I20260808 07:04:44.089564 29060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:04:44.089910 29060 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 07:04:44.092774 29060 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 07:04:44.095003 29060 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:04:44.095252 29060 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9: Loaded TSK: 0
I20260808 07:04:44.095476 29060 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:04:44.179133 29053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:44.179426 29053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:44.179481 29053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:44.180896 29053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:44.181115 29053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:44.181238 29053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:04:44.183168 29053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:46609
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=39351
--tserver_master_addrs=127.13.207.62:35665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:44.183494 29053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:44.183763 29053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:44.189213 29079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:44.189201 29080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:44.189239 29082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:44.189939 29053 server_base.cc:1034] running on GCE node
I20260808 07:04:44.190178 29053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:44.190482 29053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:44.191728 29053 hybrid_clock.cc:648] HybridClock initialized: now 1786172684191703 us; error 41 us; skew 500 ppm
Aug 08 07:04:44 dist-test-slave-8tdl krb5kdc[28389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172684, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:44.195338 29053 init.cc:377] Logged in from keytab as kudu/127.13.207.1@KRBTEST.COM (short username kudu)
I20260808 07:04:44.195953 29053 webserver.cc:460] Webserver started at http://127.13.207.1:39351/ using document root <none> and password file <none>
I20260808 07:04:44.196269 29053 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:44.196364 29053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:44.198844 29053 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:04:44.199934 29090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:44.200227 29053 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:44.200349 29053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0"
format_stamp: "Formatted at 2026-08-08 07:04:43 on dist-test-slave-8tdl"
server_key: "ab1bf00a476e707e96a93a5fe312c78d"
server_key_iv: "02c675c2c74e3b23a69e137807fa65eb"
server_key_version: "encryptionkey@0"
I20260808 07:04:44.200888 29053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:44.219165 29053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:44.222741 29053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:44.223096 29053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:44.223420 29053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:44.224246 29097 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 07:04:44.225994 29053 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 07:04:44.226073 29053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.001s sys 0.000s
I20260808 07:04:44.226114 29053 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 07:04:44.227164 29053 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 07:04:44.227219 29053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:44.227349 29097 tablet_bootstrap.cc:492] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0: Bootstrap starting.
I20260808 07:04:44.235493 29053 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:46609
I20260808 07:04:44.236057 29053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:04:44.239005 29211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:46609 every 8 connection(s)
I20260808 07:04:44.241968 29097 log.cc:824] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:44.244813 29097 tablet_bootstrap.cc:492] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:04:44.245198 29097 tablet_bootstrap.cc:492] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0: Bootstrap complete.
I20260808 07:04:44.245749 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 29053
I20260808 07:04:44.246071 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:40977
--local_ip_for_outbound_sockets=127.13.207.2
--tserver_master_addrs=127.13.207.62:35665
--webserver_port=37257
--webserver_interface=127.13.207.2
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:04:44.247310 29097 ts_tablet_manager.cc:1397] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.011s
Aug 08 07:04:44 dist-test-slave-8tdl krb5kdc[28389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172684, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:04:44.248569 29097 raft_consensus.cc:348] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } }
I20260808 07:04:44.248894 29097 raft_consensus.cc:729] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bf2d6c97e14ac5b51efbcc4e118fa0, State: Initialized, Role: FOLLOWER
I20260808 07:04:44.249076 29097 consensus_queue.cc:260] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } }
I20260808 07:04:44.249429 29097 ts_tablet_manager.cc:1428] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:04:44.254006 29215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:44.237001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:38273 (local address 127.13.207.62:35665)
0808 07:04:44.239396 (+ 2395us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:44.239406 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:44.239451 (+ 45us) server_negotiation.cc:407] Connection header received
0808 07:04:44.240357 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:44.240365 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:44.240430 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:44.240608 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:44.241812 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.243431 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.245003 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.245723 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.246533 (+ 810us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:44.249409 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:44.249414 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:44.249415 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:44.249430 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:44.251568 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:44.252177 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:44.252181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:44.252182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:44.252309 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:44.252653 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:44.252653 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:44.252654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:44.252979 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:44.253066 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:44.253500 (+ 434us) server_negotiation.cc:299] Negotiation successful
0808 07:04:44.253650 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2343,"thread_start_us":90,"threads_started":1}
I20260808 07:04:44.253937 29214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:44.236918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.1:38273)
0808 07:04:44.238878 (+ 1960us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:44.238949 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:44.240071 (+ 1122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:44.240705 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:44.240710 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:44.241339 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:44.241638 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:44.241647 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.243583 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.243585 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:44.244871 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:44.244875 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.245886 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.245888 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:44.246062 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:44.247022 (+ 960us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:44.247033 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:44.249250 (+ 2217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:44.251691 (+ 2441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:44.251702 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:44.251704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:44.252071 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:44.252403 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:44.252404 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:44.252404 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:44.252573 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:44.253058 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:44.253067 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:44.253281 (+ 214us) client_negotiation.cc:769] Sending connection context
0808 07:04:44.253432 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 07:04:44.253656 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1889,"thread_start_us":129,"threads_started":1}
I20260808 07:04:44.255422 29212 heartbeater.cc:347] Connected to a master server at 127.13.207.62:35665
I20260808 07:04:44.255548 29212 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:44.255781 29212 heartbeater.cc:511] Master 127.13.207.62:35665 requested a full tablet report, sending...
I20260808 07:04:44.256690 29011 ts_manager.cc:194] Registered new tserver with Master: c5bf2d6c97e14ac5b51efbcc4e118fa0 (127.13.207.1:46609)
I20260808 07:04:44.257956 29011 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:38273
I20260808 07:04:44.259231 29212 heartbeater.cc:503] Master 127.13.207.62:35665 was elected leader, sending a full tablet report...
I20260808 07:04:44.268061 29219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:44.257388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:46609 (local address 127.0.0.1:39820)
0808 07:04:44.262923 (+ 5535us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:44.262985 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:44.263083 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:44.263482 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:44.263486 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:44.263490 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:44.263578 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:44.263581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.265048 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.265049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:44.266534 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:44.266539 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.267669 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.267670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:44.267781 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:44.267797 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 07:04:44.267883 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 07:04:44.267952 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5489,"thread_start_us":120,"threads_started":1}
I20260808 07:04:44.268059 29220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:44.257703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39820 (local address 127.13.207.1:46609)
0808 07:04:44.263255 (+ 5552us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:44.263261 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:44.263272 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:04:44.263322 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:44.263325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:44.263343 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:44.263439 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:44.263686 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.264932 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.266652 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.267558 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.267652 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:44.267719 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:44.267917 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 07:04:44.267968 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5508,"thread_start_us":90,"threads_started":1}
I20260808 07:04:44.268958 29141 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c6b7eb400d5488281edd59397286e0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:43 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:39820
I20260808 07:04:44.269723 29221 tablet_replica.cc:330] stopping tablet replica
I20260808 07:04:44.269850 29221 raft_consensus.cc:2229] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:04:44.269927 29221 raft_consensus.cc:2258] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:04:44.270324 29221 ts_tablet_manager.cc:1905] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:04:44.272958 29221 ts_tablet_manager.cc:1918] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:04:44.273080 29221 log.cc:1196] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/wal/wals/7c6b7eb400d5488281edd59397286e0f
I20260808 07:04:44.273367 29221 ts_tablet_manager.cc:1939] T 7c6b7eb400d5488281edd59397286e0f P c5bf2d6c97e14ac5b51efbcc4e118fa0: Deleting consensus metadata
I20260808 07:04:44.274008 28995 catalog_manager.cc:4841] TS c5bf2d6c97e14ac5b51efbcc4e118fa0 (127.13.207.1:46609): tablet 7c6b7eb400d5488281edd59397286e0f (table test-table [id=12be4d82a36442a9a74a7d2d2e9de485]) successfully deleted
W20260808 07:04:44.351253 29217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:44.351557 29217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:44.351605 29217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:44.353055 29217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:44.353145 29217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:44.353199 29217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:04:44.355080 29217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:40977
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=37257
--tserver_master_addrs=127.13.207.62:35665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:44.355414 29217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:44.355697 29217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:44.361855 29228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:44.361881 29227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:44.361884 29217 server_base.cc:1034] running on GCE node
W20260808 07:04:44.361851 29230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:44.362345 29217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:44.362663 29217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:44.364104 29217 hybrid_clock.cc:648] HybridClock initialized: now 1786172684364056 us; error 64 us; skew 500 ppm
Aug 08 07:04:44 dist-test-slave-8tdl krb5kdc[28389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172684, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:44.367480 29217 init.cc:377] Logged in from keytab as kudu/127.13.207.2@KRBTEST.COM (short username kudu)
I20260808 07:04:44.368048 29217 webserver.cc:460] Webserver started at http://127.13.207.2:37257/ using document root <none> and password file <none>
I20260808 07:04:44.368322 29217 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:44.368433 29217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:44.369973 29217 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 07:04:44.370966 29238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:44.371263 29217 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:44.371536 29217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "b33e8b828e6a42ec9eb33f11aeb13838"
format_stamp: "Formatted at 2026-08-08 07:04:43 on dist-test-slave-8tdl"
server_key: "e2d53efeadb45a7a20a4488782367aea"
server_key_iv: "b626f2ae970fc0293e85e4d2cf190a83"
server_key_version: "encryptionkey@0"
I20260808 07:04:44.371977 29217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:44.387691 29217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:44.390868 29217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:44.391145 29217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:44.391494 29217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:44.392066 29245 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260808 07:04:44.393602 29245 ts_tablet_manager.cc:1852] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260808 07:04:44.393890 29245 ts_tablet_manager.cc:1905] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:04:44.396543 29245 ts_tablet_manager.cc:1918] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:04:44.396648 29245 log.cc:1196] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/wal/wals/7c6b7eb400d5488281edd59397286e0f
I20260808 07:04:44.396957 29245 ts_tablet_manager.cc:1939] T 7c6b7eb400d5488281edd59397286e0f P b33e8b828e6a42ec9eb33f11aeb13838: Deleting consensus metadata
I20260808 07:04:44.397254 29217 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260808 07:04:44.397343 29217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260808 07:04:44.397394 29217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:44.397444 29217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:44.405625 29217 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:40977
I20260808 07:04:44.405725 29366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:40977 every 8 connection(s)
I20260808 07:04:44.406102 29217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:04:44 dist-test-slave-8tdl krb5kdc[28389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172684, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:04:44.415993 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 29217
I20260808 07:04:44.416334 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:40299
--local_ip_for_outbound_sockets=127.13.207.3
--tserver_master_addrs=127.13.207.62:35665
--webserver_port=36187
--webserver_interface=127.13.207.3
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:04:44.418138 29215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:44.407013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:53569 (local address 127.13.207.62:35665)
0808 07:04:44.407104 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:44.407107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:44.407519 (+ 412us) server_negotiation.cc:407] Connection header received
0808 07:04:44.408125 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:44.408126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:44.408141 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:44.408202 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:44.409058 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.409654 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.411268 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.411497 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.411590 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:44.414342 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:44.414344 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:44.414344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:44.414353 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:44.416327 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:44.416912 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:44.416914 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:44.416915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:44.416999 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:44.417268 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:44.417269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:44.417270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:44.417438 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:44.417515 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:44.417848 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 07:04:44.417982 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 07:04:44.418231 29370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:44.407088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.2:53569)
0808 07:04:44.407310 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:44.407355 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:44.407990 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:44.408360 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:44.408364 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:44.408682 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:44.408897 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:44.408901 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.409791 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.409792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:44.411139 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:44.411143 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.411646 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.411647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:44.411779 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:44.412501 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:44.412512 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:44.414212 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:44.416486 (+ 2274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:44.416504 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:44.416508 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:44.416820 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:44.417076 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:44.417077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:44.417077 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:44.417199 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:44.417545 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:44.417553 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:44.417700 (+ 147us) client_negotiation.cc:769] Sending connection context
0808 07:04:44.417806 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 07:04:44.417985 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":99,"threads_started":1}
I20260808 07:04:44.418901 29367 heartbeater.cc:347] Connected to a master server at 127.13.207.62:35665
I20260808 07:04:44.419143 29367 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:44.419399 29367 heartbeater.cc:511] Master 127.13.207.62:35665 requested a full tablet report, sending...
I20260808 07:04:44.420074 29011 ts_manager.cc:194] Registered new tserver with Master: b33e8b828e6a42ec9eb33f11aeb13838 (127.13.207.2:40977)
I20260808 07:04:44.420552 29011 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:53569
W20260808 07:04:44.513643 29371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:44.513939 29371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:44.513988 29371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:44.515851 29371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:44.515982 29371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:44.516041 29371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:04:44.517968 29371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:40299
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=36187
--tserver_master_addrs=127.13.207.62:35665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:44.518296 29371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:44.518576 29371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:44.525460 29378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:44.525449 29376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:44.525610 29371 server_base.cc:1034] running on GCE node
W20260808 07:04:44.525449 29380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:44.526242 29371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:44.526618 29371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:44.527861 29371 hybrid_clock.cc:648] HybridClock initialized: now 1786172684527808 us; error 75 us; skew 500 ppm
Aug 08 07:04:44 dist-test-slave-8tdl krb5kdc[28389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172684, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:44.531333 29371 init.cc:377] Logged in from keytab as kudu/127.13.207.3@KRBTEST.COM (short username kudu)
I20260808 07:04:44.531920 29371 webserver.cc:460] Webserver started at http://127.13.207.3:36187/ using document root <none> and password file <none>
I20260808 07:04:44.532228 29371 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:44.532305 29371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:44.534364 29371 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:04:44.535703 29389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:44.536029 29371 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:04:44.536151 29371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "f6399499f2a242bfb972f2e51cf00a55"
format_stamp: "Formatted at 2026-08-08 07:04:43 on dist-test-slave-8tdl"
server_key: "6a9d05aa93944fc2d02362d56cd89181"
server_key_iv: "75793b2ebb7238a191ca02914f98b7d3"
server_key_version: "encryptionkey@0"
I20260808 07:04:44.536634 29371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:44.559022 29371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:44.562364 29371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:44.562722 29371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:44.563091 29371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:44.563526 29371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:44.563588 29371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:44.563634 29371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:44.563661 29371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:44.570245 29371 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:40299
I20260808 07:04:44.570403 29511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:40299 every 8 connection(s)
I20260808 07:04:44.570822 29371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:04:44.574579 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 29371
Aug 08 07:04:44 dist-test-slave-8tdl krb5kdc[28389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172684, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:04:44.583387 29517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:44.575269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33002 (local address 127.13.207.62:35665)
0808 07:04:44.575457 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:44.575459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:44.575582 (+ 123us) server_negotiation.cc:407] Connection header received
0808 07:04:44.575670 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:44.575672 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:44.575684 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:44.575779 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:44.576486 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.577036 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.578000 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.578212 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.578299 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:44.579677 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:44.579678 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:44.579679 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:44.579686 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:44.581804 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:44.582314 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:44.582315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:44.582315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:44.582388 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:44.582705 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:44.582707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:44.582707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:44.582837 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:44.583115 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:44.583161 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 07:04:44.583258 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":113,"threads_started":1}
I20260808 07:04:44.584429 29215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:44.571757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:51101 (local address 127.13.207.62:35665)
0808 07:04:44.571835 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:44.571839 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:44.572292 (+ 453us) server_negotiation.cc:407] Connection header received
0808 07:04:44.572821 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:44.572822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:44.572835 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:44.572910 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:44.574110 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.574696 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.576589 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.576967 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.577083 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:44.579868 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:44.579869 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:44.579870 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:44.580009 (+ 139us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:44.582371 (+ 2362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:44.583004 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:44.583006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:44.583006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:44.583087 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:44.583413 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:44.583414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:44.583414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:44.583548 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:44.583635 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:44.584224 (+ 589us) server_negotiation.cc:299] Negotiation successful
0808 07:04:44.584326 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:04:44.584761 29515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:44.571724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.3:51101)
0808 07:04:44.572028 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:44.572125 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:44.572655 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:44.573058 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:44.573062 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:44.573738 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:44.573978 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:44.573983 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.574981 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.574982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:44.576438 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:44.576441 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.577098 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.577099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:44.577257 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:44.577873 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:44.577882 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:44.579764 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:44.582464 (+ 2700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:44.582478 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:44.582481 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:44.582916 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:44.583172 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:44.583173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:44.583174 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:44.583315 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:44.583642 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:44.583650 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:44.583949 (+ 299us) client_negotiation.cc:769] Sending connection context
0808 07:04:44.584092 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 07:04:44.584440 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":108,"threads_started":1}
I20260808 07:04:44.585309 29512 heartbeater.cc:347] Connected to a master server at 127.13.207.62:35665
I20260808 07:04:44.585412 29512 heartbeater.cc:464] Registering TS with master...
I20260808 07:04:44.585625 29512 heartbeater.cc:511] Master 127.13.207.62:35665 requested a full tablet report, sending...
I20260808 07:04:44.586030 29011 ts_manager.cc:194] Registered new tserver with Master: f6399499f2a242bfb972f2e51cf00a55 (127.13.207.3:40299)
I20260808 07:04:44.586445 29011 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:51101
I20260808 07:04:44.595682 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:04:44.599207 29215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:44.596195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33008 (local address 127.13.207.62:35665)
0808 07:04:44.596280 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:44.596282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:44.596379 (+ 97us) server_negotiation.cc:407] Connection header received
0808 07:04:44.596499 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:44.596500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:44.596561 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:44.596657 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:04:44.596808 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.597323 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.598204 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.598418 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.598500 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:44.598727 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:04:44.598856 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:04:44.598963 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:44.599081 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 07:04:44.599119 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 07:04:44.599639 29011 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:33008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:04:44.601370 29011 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:04:44.604557 29141 tablet_service.cc:1467] Processing CreateTablet for tablet 56d74f746036497dbe055df0fa6777cc (DEFAULT_TABLE table=test-table [id=aeae182203374777a8e5781907805578]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:44.604838 29141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56d74f746036497dbe055df0fa6777cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:44.606853 29097 tablet_bootstrap.cc:492] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0: Bootstrap starting.
I20260808 07:04:44.607226 29097 tablet_bootstrap.cc:654] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:44.607824 29097 tablet_bootstrap.cc:492] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0: No bootstrap required, opened a new log
I20260808 07:04:44.607890 29097 ts_tablet_manager.cc:1397] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 07:04:44.608017 29097 raft_consensus.cc:348] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } }
I20260808 07:04:44.608114 29097 raft_consensus.cc:374] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:44.608179 29097 raft_consensus.cc:729] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bf2d6c97e14ac5b51efbcc4e118fa0, State: Initialized, Role: FOLLOWER
I20260808 07:04:44.608239 29097 consensus_queue.cc:260] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } }
I20260808 07:04:44.608352 29097 ts_tablet_manager.cc:1428] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:44.609542 29219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:44.604218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:40977 (local address 127.0.0.1:59364)
0808 07:04:44.604305 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:44.604321 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:44.604384 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:44.605204 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:44.605205 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:44.605207 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:44.605262 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:44.605264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.606831 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.606833 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:44.608030 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:44.608034 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.609257 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.609259 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:44.609332 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:44.609337 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:04:44.609396 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 07:04:44.609458 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 07:04:44.609557 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:44.604582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59364 (local address 127.13.207.2:40977)
0808 07:04:44.604824 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:44.604830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:44.604842 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:04:44.604892 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:44.604894 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:44.604912 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:44.605349 (+ 437us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:44.605426 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.606629 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.608155 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.609118 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.609232 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:44.609312 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:44.609411 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 07:04:44.609458 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":142,"threads_started":1}
I20260808 07:04:44.610183 29292 tablet_service.cc:1467] Processing CreateTablet for tablet 56d74f746036497dbe055df0fa6777cc (DEFAULT_TABLE table=test-table [id=aeae182203374777a8e5781907805578]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:44.610584 29292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56d74f746036497dbe055df0fa6777cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:44.611658 29522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:44.604336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:40299 (local address 127.0.0.1:35142)
0808 07:04:44.604506 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:44.604520 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:44.604585 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:44.607698 (+ 3113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:44.607699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:44.607701 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:44.607756 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:44.607759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.609185 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.609186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:44.610216 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:44.610219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.611324 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.611325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:44.611406 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:44.611411 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:04:44.611508 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 07:04:44.611568 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":129,"threads_started":1}
I20260808 07:04:44.611666 29524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:44.604914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35142 (local address 127.13.207.3:40299)
0808 07:04:44.607503 (+ 2589us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:44.607508 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:44.607519 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:04:44.607557 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:44.607559 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:44.607574 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:44.607699 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:44.607876 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.609055 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.610331 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.611225 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.611323 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:44.611388 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:44.611529 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 07:04:44.611574 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2537,"thread_start_us":101,"threads_started":1}
I20260808 07:04:44.612394 29441 tablet_service.cc:1467] Processing CreateTablet for tablet 56d74f746036497dbe055df0fa6777cc (DEFAULT_TABLE table=test-table [id=aeae182203374777a8e5781907805578]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:04:44.612782 29441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56d74f746036497dbe055df0fa6777cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:44.616276 29527 tablet_bootstrap.cc:492] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55: Bootstrap starting.
I20260808 07:04:44.616375 29245 tablet_bootstrap.cc:492] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838: Bootstrap starting.
I20260808 07:04:44.617033 29527 tablet_bootstrap.cc:654] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:44.617213 29245 tablet_bootstrap.cc:654] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838: Neither blocks nor log segments found. Creating new log.
I20260808 07:04:44.617604 29245 log.cc:824] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:44.617712 29527 log.cc:824] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:44.618522 29245 tablet_bootstrap.cc:492] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838: No bootstrap required, opened a new log
I20260808 07:04:44.618579 29527 tablet_bootstrap.cc:492] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55: No bootstrap required, opened a new log
I20260808 07:04:44.618607 29245 ts_tablet_manager.cc:1397] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:04:44.618659 29527 ts_tablet_manager.cc:1397] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:04:44.620118 29527 raft_consensus.cc:348] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } }
I20260808 07:04:44.620270 29527 raft_consensus.cc:374] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:44.620294 29527 raft_consensus.cc:729] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6399499f2a242bfb972f2e51cf00a55, State: Initialized, Role: FOLLOWER
I20260808 07:04:44.620203 29245 raft_consensus.cc:348] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } }
I20260808 07:04:44.620424 29245 raft_consensus.cc:374] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:04:44.620430 29527 consensus_queue.cc:260] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } }
I20260808 07:04:44.620497 29245 raft_consensus.cc:729] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b33e8b828e6a42ec9eb33f11aeb13838, State: Initialized, Role: FOLLOWER
I20260808 07:04:44.620630 29245 consensus_queue.cc:260] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } }
I20260808 07:04:44.620780 29527 ts_tablet_manager.cc:1428] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:04:44.620829 29512 heartbeater.cc:503] Master 127.13.207.62:35665 was elected leader, sending a full tablet report...
I20260808 07:04:44.620965 29245 ts_tablet_manager.cc:1428] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 07:04:44.621059 29367 heartbeater.cc:503] Master 127.13.207.62:35665 was elected leader, sending a full tablet report...
W20260808 07:04:44.656903 29369 tablet.cc:2369] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:04:44.739874 29213 tablet.cc:2369] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:04:44.766458 29530 raft_consensus.cc:482] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:04:44.766634 29530 raft_consensus.cc:504] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } }
I20260808 07:04:44.767248 29530 leader_election.cc:288] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b33e8b828e6a42ec9eb33f11aeb13838 (127.13.207.2:40977), c5bf2d6c97e14ac5b51efbcc4e118fa0 (127.13.207.1:46609)
I20260808 07:04:44.771059 29515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:44.767498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:46609 (local address 127.13.207.3:35309)
0808 07:04:44.767572 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:44.767585 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:44.767640 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:44.767930 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:44.767931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:44.767934 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:44.768019 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:44.768021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.768891 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.768892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:44.769761 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:44.769764 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.770669 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.770702 (+ 33us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:44.770815 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:44.770820 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:04:44.770894 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 07:04:44.770939 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":9}
I20260808 07:04:44.771163 29220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:44.767552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:35309 (local address 127.13.207.1:46609)
0808 07:04:44.767635 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:44.767637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:44.767651 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:04:44.767762 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:44.767763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:44.767780 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:44.767858 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:44.768121 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.768731 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.769887 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.770543 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.770632 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:44.770642 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:44.770909 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 07:04:44.771059 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 07:04:44.771178 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:44.767732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:51357 (local address 127.13.207.2:40977)
0808 07:04:44.767784 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:44.767786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:44.767909 (+ 123us) server_negotiation.cc:407] Connection header received
0808 07:04:44.767939 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:44.767940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:44.767953 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:44.768026 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:04:44.768309 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.769007 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.770184 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.770667 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.770795 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:44.770804 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:44.771038 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 07:04:44.771094 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:04:44.771229 29532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:44.767602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:40977 (local address 127.13.207.3:51357)
0808 07:04:44.767798 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:44.767809 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:44.767868 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:44.768095 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:44.768097 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:44.768099 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:04:44.768184 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:44.768186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.769113 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.769114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:44.770037 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:44.770040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.770819 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.770820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:44.770931 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:44.770938 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 07:04:44.771006 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 07:04:44.771056 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":111,"threads_started":1}
I20260808 07:04:44.771485 29162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56d74f746036497dbe055df0fa6777cc" candidate_uuid: "f6399499f2a242bfb972f2e51cf00a55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" is_pre_election: true
I20260808 07:04:44.771682 29162 raft_consensus.cc:2454] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6399499f2a242bfb972f2e51cf00a55 in term 0.
I20260808 07:04:44.771668 29316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56d74f746036497dbe055df0fa6777cc" candidate_uuid: "f6399499f2a242bfb972f2e51cf00a55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" is_pre_election: true
I20260808 07:04:44.771900 29316 raft_consensus.cc:2454] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6399499f2a242bfb972f2e51cf00a55 in term 0.
I20260808 07:04:44.771993 29392 leader_election.cc:302] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5bf2d6c97e14ac5b51efbcc4e118fa0, f6399499f2a242bfb972f2e51cf00a55; no voters:
I20260808 07:04:44.772152 29530 raft_consensus.cc:2781] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:04:44.772271 29530 raft_consensus.cc:482] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:04:44.772315 29530 raft_consensus.cc:3037] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:44.773197 29530 raft_consensus.cc:504] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } }
I20260808 07:04:44.773388 29530 leader_election.cc:288] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [CANDIDATE]: Term 1 election: Requested vote from peers b33e8b828e6a42ec9eb33f11aeb13838 (127.13.207.2:40977), c5bf2d6c97e14ac5b51efbcc4e118fa0 (127.13.207.1:46609)
I20260808 07:04:44.773643 29162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56d74f746036497dbe055df0fa6777cc" candidate_uuid: "f6399499f2a242bfb972f2e51cf00a55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0"
I20260808 07:04:44.773644 29316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56d74f746036497dbe055df0fa6777cc" candidate_uuid: "f6399499f2a242bfb972f2e51cf00a55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b33e8b828e6a42ec9eb33f11aeb13838"
I20260808 07:04:44.773751 29162 raft_consensus.cc:3037] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:44.773751 29316 raft_consensus.cc:3037] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:04:44.774987 29162 raft_consensus.cc:2454] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6399499f2a242bfb972f2e51cf00a55 in term 1.
I20260808 07:04:44.775018 29316 raft_consensus.cc:2454] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6399499f2a242bfb972f2e51cf00a55 in term 1.
I20260808 07:04:44.775254 29392 leader_election.cc:302] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5bf2d6c97e14ac5b51efbcc4e118fa0, f6399499f2a242bfb972f2e51cf00a55; no voters:
I20260808 07:04:44.775389 29530 raft_consensus.cc:2781] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:04:44.775641 29530 raft_consensus.cc:686] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [term 1 LEADER]: Becoming Leader. State: Replica: f6399499f2a242bfb972f2e51cf00a55, State: Running, Role: LEADER
I20260808 07:04:44.775779 29530 consensus_queue.cc:237] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } }
I20260808 07:04:44.776852 29011 catalog_manager.cc:5495] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6399499f2a242bfb972f2e51cf00a55 (127.13.207.3). New cstate: current_term: 1 leader_uuid: "f6399499f2a242bfb972f2e51cf00a55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } health_report { overall_health: UNKNOWN } } }
I20260808 07:04:44.795635 29524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:44.792746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35152 (local address 127.13.207.3:40299)
0808 07:04:44.792829 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:44.792831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:44.792844 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:04:44.792939 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:44.792940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:44.792970 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:44.793067 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:04:44.793225 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.793764 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.794599 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:44.794912 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:44.795005 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:44.795203 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:04:44.795322 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:04:44.795430 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:44.795469 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 07:04:44.795494 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 07:04:44.798425 29316 raft_consensus.cc:1261] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838 [term 1 FOLLOWER]: Refusing update from remote peer f6399499f2a242bfb972f2e51cf00a55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:44.798425 29162 raft_consensus.cc:1261] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 1 FOLLOWER]: Refusing update from remote peer f6399499f2a242bfb972f2e51cf00a55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:04:44.798933 29530 consensus_queue.cc:1035] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:04:44.799144 29533 consensus_queue.cc:1035] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:04:44.801600 29535 mvcc.cc:205] Tried to move back new op lower bound from 7316163316929392640 to 7316163316842291200. Current Snapshot: MvccSnapshot[applied={T|T < 7316163316929392640}]
I20260808 07:04:44.801741 29536 mvcc.cc:205] Tried to move back new op lower bound from 7316163316929392640 to 7316163316842291200. Current Snapshot: MvccSnapshot[applied={T|T < 7316163316929392640}]
I20260808 07:04:44.802958 29537 mvcc.cc:205] Tried to move back new op lower bound from 7316163316929392640 to 7316163316842291200. Current Snapshot: MvccSnapshot[applied={T|T < 7316163316929392640 or (T in {7316163316929392640})}]
I20260808 07:04:44.804504 29010 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:33008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:04:44.804625 29010 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:33008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:04:44.805486 29010 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9: Sending DeleteTablet for 3 replicas of tablet 56d74f746036497dbe055df0fa6777cc
I20260808 07:04:44.805832 29141 tablet_service.cc:1514] Processing DeleteTablet for tablet 56d74f746036497dbe055df0fa6777cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:44 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:39820
I20260808 07:04:44.805863 29292 tablet_service.cc:1514] Processing DeleteTablet for tablet 56d74f746036497dbe055df0fa6777cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:44 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:59364
I20260808 07:04:44.805948 29441 tablet_service.cc:1514] Processing DeleteTablet for tablet 56d74f746036497dbe055df0fa6777cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:04:44 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:35142
I20260808 07:04:44.806186 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 29053
I20260808 07:04:44.806201 29545 tablet_replica.cc:330] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0: stopping tablet replica
I20260808 07:04:44.807411 29547 tablet_replica.cc:330] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55: stopping tablet replica
I20260808 07:04:44.807544 29547 raft_consensus.cc:2229] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:04:44.807658 29547 raft_consensus.cc:2258] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:04:44.808377 29547 ts_tablet_manager.cc:1905] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:04:44.811128 29547 ts_tablet_manager.cc:1918] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:04:44.811368 29547 log.cc:1196] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/56d74f746036497dbe055df0fa6777cc
I20260808 07:04:44.812222 29547 ts_tablet_manager.cc:1939] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55: Deleting consensus metadata
I20260808 07:04:44.813117 28994 catalog_manager.cc:4841] TS f6399499f2a242bfb972f2e51cf00a55 (127.13.207.3:40299): tablet 56d74f746036497dbe055df0fa6777cc (table test-table [id=aeae182203374777a8e5781907805578]) successfully deleted
W20260808 07:04:44.820082 28995 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.207.1:46609 (error 108)
I20260808 07:04:44.820562 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 29217
I20260808 07:04:44.820750 29522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:44.820586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:46609 (local address 127.0.0.1:44712)
0808 07:04:44.820668 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:44.820705 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.1:46609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 07:04:44.822917 28995 catalog_manager.cc:4584] TS c5bf2d6c97e14ac5b51efbcc4e118fa0 (127.13.207.1:46609): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 56d74f746036497dbe055df0fa6777cc: Network error: Client connection negotiation failed: client connection to 127.13.207.1:46609: connect: Connection refused (error 111)
I20260808 07:04:44.827638 29522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:44.827454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:40977 (local address 127.0.0.1:59366)
0808 07:04:44.827531 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:44.827573 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.2:40977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 07:04:44.827729 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 29371
I20260808 07:04:44.835389 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 28975
I20260808 07:04:44.843685 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:35665
--webserver_interface=127.13.207.62
--webserver_port=44923
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:35665
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:04:44.936282 29549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:44.936534 29549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:44.936561 29549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:44.937975 29549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:04:44.938043 29549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:44.938058 29549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:44.938072 29549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:04:44.938086 29549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:04:44.940197 29549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:35665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:35665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=44923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:44.940562 29549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:44.940881 29549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:04:44.947049 29549 server_base.cc:1034] running on GCE node
W20260808 07:04:44.947018 29557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:44.947000 29554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:44.947011 29555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:44.947635 29549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:44.947954 29549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:44.949237 29549 hybrid_clock.cc:648] HybridClock initialized: now 1786172684949218 us; error 31 us; skew 500 ppm
Aug 08 07:04:44 dist-test-slave-8tdl krb5kdc[28389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172684, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:44.952626 29549 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:04:44.953127 29549 webserver.cc:460] Webserver started at http://127.13.207.62:44923/ using document root <none> and password file <none>
I20260808 07:04:44.953416 29549 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:44.953470 29549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:44.955212 29549 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:44.956333 29564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:44.956743 29549 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:44.956930 29549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "a389cf61f07d4008917e3985b6386ea9"
format_stamp: "Formatted at 2026-08-08 07:04:42 on dist-test-slave-8tdl"
server_key: "28a0ab6191de97b30b83102bcdf528ce"
server_key_iv: "b80c6a9c3153b5fd9527c25aecac81c7"
server_key_version: "encryptionkey@0"
I20260808 07:04:44.957584 29549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:44.977087 29549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:44.979915 29549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:44.980060 29549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:44.985649 29549 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:35665
I20260808 07:04:44.985713 29623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:35665 every 8 connection(s)
I20260808 07:04:44.986240 29549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:04:44.988415 29624 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:04:44.989182 29624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9: Bootstrap starting.
I20260808 07:04:44.991362 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 29549
I20260808 07:04:44.991736 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:46609
--local_ip_for_outbound_sockets=127.13.207.1
--tserver_master_addrs=127.13.207.62:35665
--webserver_port=39351
--webserver_interface=127.13.207.1
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:04:44.999852 29624 log.cc:824] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:45.002592 29624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:04:45.003113 29624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9: Bootstrap complete.
I20260808 07:04:45.007488 29624 raft_consensus.cc:348] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a389cf61f07d4008917e3985b6386ea9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35665 } }
I20260808 07:04:45.007804 29624 raft_consensus.cc:729] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a389cf61f07d4008917e3985b6386ea9, State: Initialized, Role: FOLLOWER
I20260808 07:04:45.008030 29624 consensus_queue.cc:260] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a389cf61f07d4008917e3985b6386ea9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35665 } }
I20260808 07:04:45.008121 29624 raft_consensus.cc:388] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:04:45.008180 29624 raft_consensus.cc:482] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:04:45.008230 29624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 2 FOLLOWER]: Advancing to term 3
I20260808 07:04:45.009433 29624 raft_consensus.cc:504] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a389cf61f07d4008917e3985b6386ea9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35665 } }
I20260808 07:04:45.009665 29624 leader_election.cc:302] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a389cf61f07d4008917e3985b6386ea9; no voters:
I20260808 07:04:45.009961 29624 leader_election.cc:288] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 07:04:45.010010 29628 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 07:04:45.010142 29628 raft_consensus.cc:686] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [term 3 LEADER]: Becoming Leader. State: Replica: a389cf61f07d4008917e3985b6386ea9, State: Running, Role: LEADER
I20260808 07:04:45.010247 29628 consensus_queue.cc:237] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a389cf61f07d4008917e3985b6386ea9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35665 } }
I20260808 07:04:45.010296 29624 sys_catalog.cc:564] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:04:45.010655 29630 sys_catalog.cc:455] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a389cf61f07d4008917e3985b6386ea9. Latest consensus state: current_term: 3 leader_uuid: "a389cf61f07d4008917e3985b6386ea9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a389cf61f07d4008917e3985b6386ea9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35665 } } }
I20260808 07:04:45.010721 29629 sys_catalog.cc:455] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a389cf61f07d4008917e3985b6386ea9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a389cf61f07d4008917e3985b6386ea9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35665 } } }
I20260808 07:04:45.010905 29630 sys_catalog.cc:458] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:45.010952 29629 sys_catalog.cc:458] T 00000000000000000000000000000000 P a389cf61f07d4008917e3985b6386ea9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:04:45.011792 29635 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:04:45.013087 29635 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:04:45.013989 29635 catalog_manager.cc:1259] Loaded cluster ID: 5abfafd5a61042a8bde82ec672ffe015
I20260808 07:04:45.014408 29635 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 07:04:45.014787 29635 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fcefece39ad google::LogMessage::Fail() at ??:0
@ 0x7fcefece688a google::LogMessage::SendToLog() at ??:0
@ 0x7fcefece34de google::LogMessage::Flush() at ??:0
@ 0x7fcefece4a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fcf06c040f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fcf06c0d8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fcf00bb10d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fcf00ba6954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fcf01a0a6db start_thread at ??:0
@ 0x7fcefd7e161f clone at ??:0
W20260808 07:04:45.115362 29626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:45.115672 29626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:45.115703 29626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:45.117283 29626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:45.117352 29626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:45.117385 29626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:04:45.120060 29626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:46609
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=39351
--tserver_master_addrs=127.13.207.62:35665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:45.120397 29626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:45.120718 29626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:45.126827 29660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:45.126871 29661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:45.127061 29626 server_base.cc:1034] running on GCE node
W20260808 07:04:45.126809 29663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:45.127450 29626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:45.127779 29626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:45.128994 29626 hybrid_clock.cc:648] HybridClock initialized: now 1786172685128958 us; error 49 us; skew 500 ppm
Aug 08 07:04:45 dist-test-slave-8tdl krb5kdc[28389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172685, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:45.132652 29626 init.cc:377] Logged in from keytab as kudu/127.13.207.1@KRBTEST.COM (short username kudu)
I20260808 07:04:45.133199 29626 webserver.cc:460] Webserver started at http://127.13.207.1:39351/ using document root <none> and password file <none>
I20260808 07:04:45.133486 29626 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:45.133564 29626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:45.135246 29626 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:04:45.136245 29671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:45.136600 29626 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:45.136734 29626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0"
format_stamp: "Formatted at 2026-08-08 07:04:43 on dist-test-slave-8tdl"
server_key: "ab1bf00a476e707e96a93a5fe312c78d"
server_key_iv: "02c675c2c74e3b23a69e137807fa65eb"
server_key_version: "encryptionkey@0"
I20260808 07:04:45.137262 29626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:45.151959 29626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:45.155179 29626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:45.155391 29626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:45.155723 29626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:45.156241 29679 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 07:04:45.158399 29626 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 07:04:45.158502 29626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 07:04:45.158566 29626 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 07:04:45.159789 29626 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 07:04:45.159865 29626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 07:04:45.159950 29679 tablet_bootstrap.cc:492] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0: Bootstrap starting.
I20260808 07:04:45.169672 29626 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:46609
I20260808 07:04:45.169862 29792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:46609 every 8 connection(s)
I20260808 07:04:45.170186 29626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:04:45.175539 29679 log.cc:824] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:45.179194 29679 tablet_bootstrap.cc:492] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:04:45.179675 29679 tablet_bootstrap.cc:492] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0: Bootstrap complete.
I20260808 07:04:45.179909 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 29626
I20260808 07:04:45.180251 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:40977
--local_ip_for_outbound_sockets=127.13.207.2
--tserver_master_addrs=127.13.207.62:35665
--webserver_port=37257
--webserver_interface=127.13.207.2
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:04:45.181373 29679 ts_tablet_manager.cc:1397] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0: Time spent bootstrapping tablet: real 0.022s user 0.005s sys 0.014s
I20260808 07:04:45.183816 29679 raft_consensus.cc:348] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } }
I20260808 07:04:45.184626 29679 raft_consensus.cc:729] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bf2d6c97e14ac5b51efbcc4e118fa0, State: Initialized, Role: FOLLOWER
I20260808 07:04:45.184932 29679 consensus_queue.cc:260] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } }
I20260808 07:04:45.185354 29679 ts_tablet_manager.cc:1428] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0: Time spent starting tablet: real 0.004s user 0.007s sys 0.000s
W20260808 07:04:45.278426 29796 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:04:45.171144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.1:60425)
0808 07:04:45.171497 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:45.171544 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:45.172163 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:45.278212 (+106049us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":152,"threads_started":1}
W20260808 07:04:45.278854 29793 heartbeater.cc:644] Failed to heartbeat to 127.13.207.62:35665 (0 consecutive failures): Network error: Failed to ping master at 127.13.207.62:35665: Client connection negotiation failed: client connection to 127.13.207.62:35665: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 07:04:45.279250 29796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:45.279116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.1:47219)
0808 07:04:45.279176 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:45.279227 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 07:04:45.279750 29796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:45.279613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.1:47139)
0808 07:04:45.279690 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:45.279725 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 07:04:45.280030 29793 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:04:45.287911 29798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:45.288168 29798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:45.288200 29798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:45.289683 29798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:45.289747 29798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:45.289783 29798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:04:45.291704 29798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:40977
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=37257
--tserver_master_addrs=127.13.207.62:35665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:45.291961 29798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:45.292194 29798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:45.298473 29804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:45.298557 29806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:45.298616 29798 server_base.cc:1034] running on GCE node
W20260808 07:04:45.298486 29808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:45.299055 29798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:45.299382 29798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:45.300618 29798 hybrid_clock.cc:648] HybridClock initialized: now 1786172685300580 us; error 56 us; skew 500 ppm
Aug 08 07:04:45 dist-test-slave-8tdl krb5kdc[28389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172685, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:45.304039 29798 init.cc:377] Logged in from keytab as kudu/127.13.207.2@KRBTEST.COM (short username kudu)
I20260808 07:04:45.304734 29798 webserver.cc:460] Webserver started at http://127.13.207.2:37257/ using document root <none> and password file <none>
I20260808 07:04:45.305039 29798 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:45.305099 29798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:45.307098 29798 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:04:45.308205 29817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:45.308527 29798 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:45.308647 29798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "b33e8b828e6a42ec9eb33f11aeb13838"
format_stamp: "Formatted at 2026-08-08 07:04:43 on dist-test-slave-8tdl"
server_key: "e2d53efeadb45a7a20a4488782367aea"
server_key_iv: "b626f2ae970fc0293e85e4d2cf190a83"
server_key_version: "encryptionkey@0"
I20260808 07:04:45.309128 29798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:45.325989 29798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:45.328897 29798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:45.329083 29798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:45.329344 29798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:45.329799 29824 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 07:04:45.331575 29798 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 07:04:45.331650 29798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 07:04:45.331681 29798 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 07:04:45.332541 29798 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 07:04:45.332585 29798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 07:04:45.332710 29824 tablet_bootstrap.cc:492] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838: Bootstrap starting.
I20260808 07:04:45.341454 29798 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:40977
I20260808 07:04:45.341531 29937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:40977 every 8 connection(s)
I20260808 07:04:45.341959 29798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:04:45.343408 29940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:45.342870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.2:34859)
0808 07:04:45.343084 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:45.343196 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":90,"threads_started":1}
W20260808 07:04:45.343765 29938 heartbeater.cc:644] Failed to heartbeat to 127.13.207.62:35665 (0 consecutive failures): Network error: Failed to ping master at 127.13.207.62:35665: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
I20260808 07:04:45.344152 29940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:45.344026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.2:53407)
0808 07:04:45.344084 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:45.344120 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 07:04:45.344550 29940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:45.344418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.2:58341)
0808 07:04:45.344489 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:45.344525 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 07:04:45.344731 29938 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:04:45.344714 29824 log.cc:824] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:45.347329 29824 tablet_bootstrap.cc:492] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:04:45.347608 29824 tablet_bootstrap.cc:492] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838: Bootstrap complete.
I20260808 07:04:45.349037 29824 ts_tablet_manager.cc:1397] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.009s
I20260808 07:04:45.349427 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 29798
I20260808 07:04:45.349710 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:40299
--local_ip_for_outbound_sockets=127.13.207.3
--tserver_master_addrs=127.13.207.62:35665
--webserver_port=36187
--webserver_interface=127.13.207.3
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:44061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:04:45.351615 29824 raft_consensus.cc:348] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } }
I20260808 07:04:45.351989 29824 raft_consensus.cc:729] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b33e8b828e6a42ec9eb33f11aeb13838, State: Initialized, Role: FOLLOWER
I20260808 07:04:45.352139 29824 consensus_queue.cc:260] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } }
I20260808 07:04:45.352421 29824 ts_tablet_manager.cc:1428] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838: Time spent starting tablet: real 0.003s user 0.006s sys 0.000s
I20260808 07:04:45.352865 29940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:45.352709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.2:56939)
0808 07:04:45.352796 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:45.352837 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 07:04:45.353317 29940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:45.353200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.2:59527)
0808 07:04:45.353256 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:45.353286 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 07:04:45.445660 29943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:04:45.445906 29943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:04:45.445936 29943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:04:45.447482 29943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:04:45.447539 29943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:04:45.447571 29943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:04:45.449267 29943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:44061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:40299
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=36187
--tserver_master_addrs=127.13.207.62:35665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:04:45.449558 29943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:04:45.449867 29943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:04:45.455894 29950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:45.455899 29952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:04:45.455899 29949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:04:45.456331 29943 server_base.cc:1034] running on GCE node
I20260808 07:04:45.456655 29943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:04:45.456969 29943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:04:45.458180 29943 hybrid_clock.cc:648] HybridClock initialized: now 1786172685458167 us; error 40 us; skew 500 ppm
Aug 08 07:04:45 dist-test-slave-8tdl krb5kdc[28389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172685, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:04:45.461090 29943 init.cc:377] Logged in from keytab as kudu/127.13.207.3@KRBTEST.COM (short username kudu)
I20260808 07:04:45.461535 29943 webserver.cc:460] Webserver started at http://127.13.207.3:36187/ using document root <none> and password file <none>
I20260808 07:04:45.461788 29943 fs_manager.cc:359] Metadata directory not provided
I20260808 07:04:45.461839 29943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:04:45.463567 29943 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:04:45.464432 29960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:45.464670 29943 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:04:45.464789 29943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "f6399499f2a242bfb972f2e51cf00a55"
format_stamp: "Formatted at 2026-08-08 07:04:43 on dist-test-slave-8tdl"
server_key: "6a9d05aa93944fc2d02362d56cd89181"
server_key_iv: "75793b2ebb7238a191ca02914f98b7d3"
server_key_version: "encryptionkey@0"
I20260808 07:04:45.465191 29943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:04:45.483176 29943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:04:45.485807 29943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:04:45.486006 29943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:04:45.486276 29943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:04:45.486611 29943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:04:45.486670 29943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:45.486734 29943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:04:45.486769 29943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:04:45.493805 29943 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:40299
I20260808 07:04:45.494009 30078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:40299 every 8 connection(s)
I20260808 07:04:45.494347 29943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:04:45.495728 30081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:45.495234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.3:38103)
0808 07:04:45.495438 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:45.495549 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":75,"threads_started":1}
W20260808 07:04:45.496351 30079 heartbeater.cc:644] Failed to heartbeat to 127.13.207.62:35665 (0 consecutive failures): Network error: Failed to ping master at 127.13.207.62:35665: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
I20260808 07:04:45.496816 30081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:45.496703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.3:35499)
0808 07:04:45.496759 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:45.496793 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 07:04:45.497403 30081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:45.497108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.3:36755)
0808 07:04:45.497315 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:45.497357 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
W20260808 07:04:45.497615 30079 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:04:45.497624 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 29943
I20260808 07:04:46.280808 30083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:46.280520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.1:42761)
0808 07:04:46.280692 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:46.280741 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":75,"threads_started":1}
I20260808 07:04:46.354334 30084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:46.354021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.2:55899)
0808 07:04:46.354209 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:46.354262 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":92,"threads_started":1}
I20260808 07:04:46.498337 30085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:46.498081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.3:45223)
0808 07:04:46.498229 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:46.498268 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260808 07:04:46.994042 30086 raft_consensus.cc:482] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:04:46.994186 30086 raft_consensus.cc:504] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } }
I20260808 07:04:46.994556 30086 leader_election.cc:288] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b33e8b828e6a42ec9eb33f11aeb13838 (127.13.207.2:40977), f6399499f2a242bfb972f2e51cf00a55 (127.13.207.3:40299)
Aug 08 07:04:46 dist-test-slave-8tdl krb5kdc[28389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172685, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.2@KRBTEST.COM
Aug 08 07:04:46 dist-test-slave-8tdl krb5kdc[28389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172685, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.3@KRBTEST.COM
I20260808 07:04:47.008441 30087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:46.994757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:40977 (local address 127.13.207.1:53861)
0808 07:04:46.994959 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:46.994973 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:46.995113 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:46.995920 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:46.995923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:46.996385 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:46.996683 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:46.996691 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:46.998012 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:46.998013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:46.998894 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:46.998896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:47.000010 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:47.000012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:47.000100 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:47.000526 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:47.000532 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:47.002306 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:47.005569 (+ 3263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:47.005576 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:47.005579 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:47.006353 (+ 774us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:47.006793 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:47.006794 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:47.006795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:47.007005 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:47.007651 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:47.007667 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:47.007861 (+ 194us) client_negotiation.cc:769] Sending connection context
0808 07:04:47.008014 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 07:04:47.008317 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":208,"thread_start_us":71,"threads_started":1}
I20260808 07:04:47.008605 30090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:46.995302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:51787 (local address 127.13.207.3:40299)
0808 07:04:46.995526 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:46.995534 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:46.995563 (+ 29us) server_negotiation.cc:407] Connection header received
0808 07:04:46.996227 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:46.996230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:46.996284 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:46.996449 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:46.996961 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:46.997841 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:46.998989 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:46.999453 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:46.999544 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:47.002938 (+ 3394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:47.002943 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:47.002944 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:47.002957 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:47.005368 (+ 2411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:47.006507 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:47.006510 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:47.006511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:47.006705 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:47.007138 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:47.007140 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:47.007140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:47.007539 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:47.007642 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:47.008205 (+ 563us) server_negotiation.cc:299] Negotiation successful
0808 07:04:47.008382 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":127,"threads_started":1}
I20260808 07:04:47.008841 30089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:46.995249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:53861 (local address 127.13.207.2:40977)
0808 07:04:46.995492 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:46.995503 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:46.995525 (+ 22us) server_negotiation.cc:407] Connection header received
0808 07:04:46.995632 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:46.995634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:46.995675 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:46.995789 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:46.996913 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:46.997857 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:46.999253 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:46.999811 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:47.000025 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:47.002728 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:47.002734 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:47.002735 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:47.002753 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:47.005406 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:47.006488 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:47.006493 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:47.006494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:47.006712 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:47.007139 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:47.007140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:47.007141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:47.007538 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:47.007642 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:47.008229 (+ 587us) server_negotiation.cc:299] Negotiation successful
0808 07:04:47.008530 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":103,"threads_started":1}
I20260808 07:04:47.008441 30088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:46.994896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:40299 (local address 127.13.207.1:51787)
0808 07:04:46.995014 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:46.995155 (+ 141us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:46.995221 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:46.996443 (+ 1222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:46.996445 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:46.996631 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:46.996692 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:46.996694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:46.998003 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:46.998005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:46.998893 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:46.998896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:46.999564 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:46.999565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:46.999687 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:47.000399 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:47.000408 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:47.002792 (+ 2384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:47.005557 (+ 2765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:47.005576 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:47.005579 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:47.006354 (+ 775us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:47.006806 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:47.006807 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:47.006808 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:47.007005 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:47.007651 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:47.007667 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:47.007861 (+ 194us) client_negotiation.cc:769] Sending connection context
0808 07:04:47.008014 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 07:04:47.008259 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":11,"thread_start_us":56,"threads_started":1}
I20260808 07:04:47.009371 29890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56d74f746036497dbe055df0fa6777cc" candidate_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" is_pre_election: true
I20260808 07:04:47.009560 29890 raft_consensus.cc:2454] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bf2d6c97e14ac5b51efbcc4e118fa0 in term 1.
I20260808 07:04:47.009859 29672 leader_election.cc:302] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b33e8b828e6a42ec9eb33f11aeb13838, c5bf2d6c97e14ac5b51efbcc4e118fa0; no voters:
I20260808 07:04:47.010094 30086 raft_consensus.cc:2781] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260808 07:04:47.010146 29672 leader_election.cc:341] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6399499f2a242bfb972f2e51cf00a55 (127.13.207.3:40299): Not found: Tablet not found: 56d74f746036497dbe055df0fa6777cc
I20260808 07:04:47.010175 30086 raft_consensus.cc:482] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:04:47.010202 30086 raft_consensus.cc:3037] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:04:47.011543 30086 raft_consensus.cc:504] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } }
I20260808 07:04:47.011713 30086 leader_election.cc:288] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [CANDIDATE]: Term 2 election: Requested vote from peers b33e8b828e6a42ec9eb33f11aeb13838 (127.13.207.2:40977), f6399499f2a242bfb972f2e51cf00a55 (127.13.207.3:40299)
I20260808 07:04:47.012166 29890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56d74f746036497dbe055df0fa6777cc" candidate_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b33e8b828e6a42ec9eb33f11aeb13838"
W20260808 07:04:47.012238 29672 leader_election.cc:341] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer f6399499f2a242bfb972f2e51cf00a55 (127.13.207.3:40299): Not found: Tablet not found: 56d74f746036497dbe055df0fa6777cc
I20260808 07:04:47.012292 29890 raft_consensus.cc:3037] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838 [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:04:47.013764 29890 raft_consensus.cc:2454] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bf2d6c97e14ac5b51efbcc4e118fa0 in term 2.
I20260808 07:04:47.014079 29672 leader_election.cc:302] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b33e8b828e6a42ec9eb33f11aeb13838, c5bf2d6c97e14ac5b51efbcc4e118fa0; no voters: f6399499f2a242bfb972f2e51cf00a55
I20260808 07:04:47.014235 30086 raft_consensus.cc:2781] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 07:04:47.014374 30086 raft_consensus.cc:686] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [term 2 LEADER]: Becoming Leader. State: Replica: c5bf2d6c97e14ac5b51efbcc4e118fa0, State: Running, Role: LEADER
I20260808 07:04:47.014465 30086 consensus_queue.cc:237] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } }
I20260808 07:04:47.015057 30088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:47.014775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.1:32897)
0808 07:04:47.014971 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:47.015016 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
I20260808 07:04:47.355350 30093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:47.355090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.2:50573)
0808 07:04:47.355244 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:47.355294 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":76,"threads_started":1}
W20260808 07:04:47.397794 29672 consensus_peers.cc:487] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 -> Peer f6399499f2a242bfb972f2e51cf00a55 (127.13.207.3:40299): Couldn't send request to peer f6399499f2a242bfb972f2e51cf00a55. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 56d74f746036497dbe055df0fa6777cc. This is attempt 1: this message will repeat every 5th retry.
I20260808 07:04:47.499404 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:47.499066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.3:32821)
0808 07:04:47.499274 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:47.499323 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":85,"threads_started":1}
I20260808 07:04:47.521422 29890 raft_consensus.cc:1261] T 56d74f746036497dbe055df0fa6777cc P b33e8b828e6a42ec9eb33f11aeb13838 [term 2 FOLLOWER]: Refusing update from remote peer c5bf2d6c97e14ac5b51efbcc4e118fa0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260808 07:04:47.522094 30086 consensus_queue.cc:1035] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260808 07:04:47.523265 30093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:47.523130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.2:56741)
0808 07:04:47.523190 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:47.523231 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 07:04:47.524154 30100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:47.523945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.1:35257)
0808 07:04:47.524064 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:47.524099 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
I20260808 07:04:47.524260 30093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:47.524126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.2:53887)
0808 07:04:47.524165 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:47.524209 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 07:04:47.955858 30101 ts_tablet_manager.cc:927] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55: Initiating tablet copy from peer c5bf2d6c97e14ac5b51efbcc4e118fa0 (127.13.207.1:46609)
I20260808 07:04:47.956207 30101 tablet_copy_client.cc:316] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55: tablet copy: Beginning tablet copy session from remote peer at address 127.13.207.1:46609
Aug 08 07:04:47 dist-test-slave-8tdl krb5kdc[28389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172685, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for kudu/127.13.207.1@KRBTEST.COM
I20260808 07:04:47.970079 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:47.956582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:46609 (local address 127.13.207.3:46897)
0808 07:04:47.956689 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:47.956716 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 07:04:47.956811 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:04:47.957200 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:04:47.957208 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:04:47.957578 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:04:47.957659 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:47.957661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:47.958814 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:47.958815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:47.959460 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:04:47.959463 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:04:47.959954 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:04:47.959955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:04:47.960048 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:47.964163 (+ 4115us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:04:47.964169 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:04:47.965943 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:04:47.968292 (+ 2349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:47.968306 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:47.968310 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:47.968712 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:47.969148 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:04:47.969149 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:04:47.969149 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:04:47.969230 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:04:47.969695 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:04:47.969705 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:04:47.969769 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 07:04:47.969862 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 07:04:47.969998 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 07:04:47.970350 30106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:47.956711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:46897 (local address 127.13.207.1:46609)
0808 07:04:47.956952 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 07:04:47.956957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:04:47.957027 (+ 70us) server_negotiation.cc:407] Connection header received
0808 07:04:47.957056 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:04:47.957058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:04:47.957072 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:04:47.957157 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:04:47.957769 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:47.958731 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:47.959546 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:04:47.959854 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:04:47.959917 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:04:47.966081 (+ 6164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:04:47.966102 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:04:47.966102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:04:47.966114 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:04:47.968134 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:47.968819 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:47.968824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:47.968824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:47.968987 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:04:47.969348 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:04:47.969349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:04:47.969350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:04:47.969619 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:04:47.969693 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:04:47.970080 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 07:04:47.970222 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":156,"threads_started":1}
I20260808 07:04:47.970851 29747 tablet_copy_service.cc:140] P c5bf2d6c97e14ac5b51efbcc4e118fa0: Received BeginTabletCopySession request for tablet 56d74f746036497dbe055df0fa6777cc from peer f6399499f2a242bfb972f2e51cf00a55 ({username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:46897)
I20260808 07:04:47.970935 29747 tablet_copy_service.cc:161] P c5bf2d6c97e14ac5b51efbcc4e118fa0: Beginning new tablet copy session on tablet 56d74f746036497dbe055df0fa6777cc from peer f6399499f2a242bfb972f2e51cf00a55 at {username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:46897: session id = f6399499f2a242bfb972f2e51cf00a55-56d74f746036497dbe055df0fa6777cc
I20260808 07:04:47.971467 29747 tablet_copy_source_session.cc:205] T 56d74f746036497dbe055df0fa6777cc P c5bf2d6c97e14ac5b51efbcc4e118fa0: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 07:04:47.972452 30101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56d74f746036497dbe055df0fa6777cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:04:47.975291 30101 tablet_copy_client.cc:791] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55: tablet copy: Starting download of 0 data blocks...
I20260808 07:04:47.975476 30101 tablet_copy_client.cc:655] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55: tablet copy: Starting download of 1 WAL segments...
I20260808 07:04:47.976650 30101 tablet_copy_client.cc:531] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 07:04:47.977481 30101 tablet_bootstrap.cc:492] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55: Bootstrap starting.
I20260808 07:04:47.979074 30101 log.cc:824] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55: Log is configured to *not* fsync() on all Append() calls
I20260808 07:04:47.982050 30101 tablet_bootstrap.cc:492] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:04:47.982347 30101 tablet_bootstrap.cc:492] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55: Bootstrap complete.
I20260808 07:04:47.982529 30101 ts_tablet_manager.cc:1397] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 07:04:47.983737 30101 raft_consensus.cc:348] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } }
I20260808 07:04:47.983943 30101 raft_consensus.cc:729] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6399499f2a242bfb972f2e51cf00a55, State: Initialized, Role: FOLLOWER
I20260808 07:04:47.984097 30101 consensus_queue.cc:260] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b33e8b828e6a42ec9eb33f11aeb13838" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 40977 } } peers { permanent_uuid: "f6399499f2a242bfb972f2e51cf00a55" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 40299 } } peers { permanent_uuid: "c5bf2d6c97e14ac5b51efbcc4e118fa0" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46609 } }
I20260808 07:04:47.984424 30101 ts_tablet_manager.cc:1428] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20260808 07:04:47.984903 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:47.984733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.3:58219)
0808 07:04:47.984816 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:47.984868 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 07:04:47.985075 29747 tablet_copy_service.cc:342] P c5bf2d6c97e14ac5b51efbcc4e118fa0: Request end of tablet copy session f6399499f2a242bfb972f2e51cf00a55-56d74f746036497dbe055df0fa6777cc received from {username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:46897
I20260808 07:04:47.985159 29747 tablet_copy_service.cc:433] P c5bf2d6c97e14ac5b51efbcc4e118fa0: ending tablet copy session f6399499f2a242bfb972f2e51cf00a55-56d74f746036497dbe055df0fa6777cc on tablet 56d74f746036497dbe055df0fa6777cc with peer f6399499f2a242bfb972f2e51cf00a55
I20260808 07:04:47.985332 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:47.985217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.3:37757)
0808 07:04:47.985260 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:47.985302 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260808 07:04:48.525059 30109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:48.524697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.1:39503)
0808 07:04:48.524901 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:48.524959 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":57,"threads_started":1}
I20260808 07:04:48.525460 30110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:48.524735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.2:51095)
0808 07:04:48.525341 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:48.525401 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":244,"threads_started":1}
I20260808 07:04:48.562491 30109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:48.562323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.1:50061)
0808 07:04:48.562393 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:48.562441 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 07:04:48.562628 30031 raft_consensus.cc:1203] T 56d74f746036497dbe055df0fa6777cc P f6399499f2a242bfb972f2e51cf00a55 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260808 07:04:48.563529 30113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:48.563152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.3:33327)
0808 07:04:48.563385 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:48.563465 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":74,"threads_started":1}
I20260808 07:04:49.526372 30114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:49.526129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.2:39995)
0808 07:04:49.526277 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:49.526323 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
I20260808 07:04:49.563333 30115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:49.563096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.1:39759)
0808 07:04:49.563234 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:49.563281 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260808 07:04:49.564327 30116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:49.564111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.3:33845)
0808 07:04:49.564246 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:49.564288 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":57,"threads_started":1}
I20260808 07:04:50.527398 30117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:50.527096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.2:34101)
0808 07:04:50.527271 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:50.527315 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":74,"threads_started":1}
I20260808 07:04:50.564409 30119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:50.564067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.1:36539)
0808 07:04:50.564251 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:50.564312 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":111,"threads_started":1}
I20260808 07:04:50.565299 30119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:50.565109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.1:36431)
0808 07:04:50.565206 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:50.565257 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 07:04:50.565312 30120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:50.565034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.3:52309)
0808 07:04:50.565228 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:50.565267 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":84,"threads_started":1}
I20260808 07:04:51.528345 30121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:51.528052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.2:42981)
0808 07:04:51.528243 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:51.528294 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":72,"threads_started":1}
I20260808 07:04:51.566252 30123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:51.565984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.1:48729)
0808 07:04:51.566166 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:51.566200 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":86,"threads_started":1}
I20260808 07:04:51.566252 30122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:51.565984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.3:59161)
0808 07:04:51.566148 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:51.566204 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":83,"threads_started":1}
I20260808 07:04:52.529367 30124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:52.529055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.2:36037)
0808 07:04:52.529256 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:52.529312 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":101,"threads_started":1}
I20260808 07:04:52.567212 30125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:52.566914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.3:60695)
0808 07:04:52.567095 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:52.567134 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":100,"threads_started":1}
I20260808 07:04:52.567241 30126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:52.566914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.1:33267)
0808 07:04:52.567099 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:52.567182 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":109,"threads_started":1}
I20260808 07:04:53.530427 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.530117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.2:37913)
0808 07:04:53.530317 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:53.530373 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":72,"threads_started":1}
I20260808 07:04:53.568003 30129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.567750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.1:49697)
0808 07:04:53.567903 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:53.567935 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":70,"threads_started":1}
I20260808 07:04:53.567993 30128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:53.567749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.3:56729)
0808 07:04:53.567897 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:53.567938 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260808 07:04:54.531544 30130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:54.531230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.2:56569)
0808 07:04:54.531435 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:54.531488 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":75,"threads_started":1}
I20260808 07:04:54.569118 30131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:54.568691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.1:50019)
0808 07:04:54.568990 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:54.569044 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":97,"threads_started":1}
I20260808 07:04:54.569375 30132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:54.568967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.3:56221)
0808 07:04:54.569204 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:54.569313 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":77,"threads_started":1}
I20260808 07:04:55.532658 30133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:55.532330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.2:38645)
0808 07:04:55.532546 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:55.532607 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":101,"threads_started":1}
I20260808 07:04:55.570191 30134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:55.569856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.1:53127)
0808 07:04:55.570087 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:55.570137 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":100,"threads_started":1}
I20260808 07:04:55.570451 30136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:55.570008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.3:56927)
0808 07:04:55.570166 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:55.570391 (+ 225us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
I20260808 07:04:56.533690 30138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.533385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.2:55499)
0808 07:04:56.533590 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:56.533639 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":90,"threads_started":1}
I20260808 07:04:56.571236 30139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.570963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.1:33751)
0808 07:04:56.571136 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:56.571185 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":108,"threads_started":1}
I20260808 07:04:56.571422 30140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:56.571158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.3:54571)
0808 07:04:56.571307 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:56.571363 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
I20260808 07:04:57.534921 30141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:57.534502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.2:34489)
0808 07:04:57.534806 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:57.534867 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":80,"threads_started":1}
I20260808 07:04:57.572129 30142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:57.571840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.1:60391)
0808 07:04:57.572000 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:57.572058 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260808 07:04:57.572552 30143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:57.572088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.3:49549)
0808 07:04:57.572391 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:57.572485 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":74,"threads_started":1}
I20260808 07:04:58.536019 30144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:58.535719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.2:52077)
0808 07:04:58.535898 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:58.535962 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":76,"threads_started":1}
I20260808 07:04:58.573254 30145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:58.572868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.1:44159)
0808 07:04:58.573080 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:58.573177 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":88,"threads_started":1}
I20260808 07:04:58.573650 30146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:58.573302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.3:49521)
0808 07:04:58.573503 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:58.573595 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":93,"threads_started":1}
I20260808 07:04:59.537041 30148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:59.536749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.2:56879)
0808 07:04:59.536936 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:59.536988 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":83,"threads_started":1}
I20260808 07:04:59.574532 30149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:59.574172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.1:56801)
0808 07:04:59.574328 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:59.574443 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":77,"threads_started":1}
I20260808 07:04:59.574556 30150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:04:59.574318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35665 (local address 127.13.207.3:38401)
0808 07:04:59.574460 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 07:04:59.574504 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
I20260808 07:05:00.499723 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 29626
I20260808 07:05:00.507284 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 29798
I20260808 07:05:00.514844 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 29943
2026-08-08T07:05:00Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18958 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:05:00 dist-test-slave-8tdl krb5kdc[30154](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:05:00 dist-test-slave-8tdl krb5kdc[30154](info): set up 2 sockets
Aug 08 07:05:00 dist-test-slave-8tdl krb5kdc[30154](info): commencing operation
krb5kdc: starting...
W20260808 07:05:01.727450 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:05:01 dist-test-slave-8tdl krb5kdc[30154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:05:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:05:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:05:01.896039 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:35651
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:35651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:05:01.996642 30168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:05:01.996954 30168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:05:01.997008 30168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:05:01.998322 30168 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 07:05:01.998406 30168 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 07:05:01.998432 30168 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 07:05:01.998450 30168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:05:01.998473 30168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:05:01.998515 30168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:05:01.998536 30168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:05:01.998564 30168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:05:02.000372 30168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:35651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:35651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:05:02.000711 30168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:05:02.001112 30168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:05:02.009091 30168 server_base.cc:1034] running on GCE node
W20260808 07:05:02.010048 30173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:05:02.010284 30176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:05:02.010284 30174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:05:02.010967 30168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:05:02.011695 30168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:05:02.013026 30168 hybrid_clock.cc:648] HybridClock initialized: now 1786172702012998 us; error 50 us; skew 500 ppm
Aug 08 07:05:02 dist-test-slave-8tdl krb5kdc[30154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172702, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:05:02.023156 30168 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:05:02.023845 30168 webserver.cc:460] Webserver started at http://127.13.207.62:45219/ using document root <none> and password file <none>
I20260808 07:05:02.024307 30168 fs_manager.cc:359] Metadata directory not provided
I20260808 07:05:02.024446 30168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:05:02.024658 30168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:05:02.027307 30168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "b383a3ee06cc4e09bd96131a55c522e9"
format_stamp: "Formatted at 2026-08-08 07:05:02 on dist-test-slave-8tdl"
server_key: "a44622e8bfe451ff2733e7c559433814"
server_key_iv: "4bc7fdaad653a1950f550c11d9a5f642"
server_key_version: "encryptionkey@0"
I20260808 07:05:02.027830 30168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "b383a3ee06cc4e09bd96131a55c522e9"
format_stamp: "Formatted at 2026-08-08 07:05:02 on dist-test-slave-8tdl"
server_key: "a44622e8bfe451ff2733e7c559433814"
server_key_iv: "4bc7fdaad653a1950f550c11d9a5f642"
server_key_version: "encryptionkey@0"
I20260808 07:05:02.029634 30168 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 07:05:02.031246 30183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:02.031788 30168 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260808 07:05:02.032063 30168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "b383a3ee06cc4e09bd96131a55c522e9"
format_stamp: "Formatted at 2026-08-08 07:05:02 on dist-test-slave-8tdl"
server_key: "a44622e8bfe451ff2733e7c559433814"
server_key_iv: "4bc7fdaad653a1950f550c11d9a5f642"
server_key_version: "encryptionkey@0"
I20260808 07:05:02.032182 30168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:05:02.058125 30168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:05:02.061034 30168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:05:02.061455 30168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:05:02.067616 30168 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:35651
I20260808 07:05:02.067713 30238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:35651 every 8 connection(s)
I20260808 07:05:02.068152 30168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:05:02.069365 30240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:05:02.071887 30240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9: Bootstrap starting.
I20260808 07:05:02.072546 30240 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9: Neither blocks nor log segments found. Creating new log.
I20260808 07:05:02.072810 30240 log.cc:824] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9: Log is configured to *not* fsync() on all Append() calls
I20260808 07:05:02.073638 30240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9: No bootstrap required, opened a new log
I20260808 07:05:02.074172 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 30168
I20260808 07:05:02.074273 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:05:02.074542 14140 external_mini_cluster.cc:1428] Setting key 8e6c08c295ce7bd50d19cdef7369123e
I20260808 07:05:02.076100 30240 raft_consensus.cc:348] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b383a3ee06cc4e09bd96131a55c522e9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35651 } }
I20260808 07:05:02.076303 30240 raft_consensus.cc:374] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:05:02.076344 30240 raft_consensus.cc:729] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b383a3ee06cc4e09bd96131a55c522e9, State: Initialized, Role: FOLLOWER
I20260808 07:05:02.076526 30240 consensus_queue.cc:260] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b383a3ee06cc4e09bd96131a55c522e9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35651 } }
I20260808 07:05:02.076622 30240 raft_consensus.cc:388] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:05:02.076646 30240 raft_consensus.cc:482] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:05:02.076676 30240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:05:02.077912 30240 raft_consensus.cc:504] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b383a3ee06cc4e09bd96131a55c522e9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35651 } }
I20260808 07:05:02.078039 30240 leader_election.cc:302] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b383a3ee06cc4e09bd96131a55c522e9; no voters:
I20260808 07:05:02.078212 30240 leader_election.cc:288] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:05:02.078308 30245 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:05:02.078497 30240 sys_catalog.cc:564] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:05:02.078552 30245 raft_consensus.cc:686] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 1 LEADER]: Becoming Leader. State: Replica: b383a3ee06cc4e09bd96131a55c522e9, State: Running, Role: LEADER
I20260808 07:05:02.078735 30245 consensus_queue.cc:237] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b383a3ee06cc4e09bd96131a55c522e9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35651 } }
I20260808 07:05:02.080235 30246 sys_catalog.cc:455] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b383a3ee06cc4e09bd96131a55c522e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b383a3ee06cc4e09bd96131a55c522e9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35651 } } }
I20260808 07:05:02.080329 30246 sys_catalog.cc:458] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:05:02.080513 30247 sys_catalog.cc:455] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b383a3ee06cc4e09bd96131a55c522e9. Latest consensus state: current_term: 1 leader_uuid: "b383a3ee06cc4e09bd96131a55c522e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b383a3ee06cc4e09bd96131a55c522e9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35651 } } }
I20260808 07:05:02.080667 30247 sys_catalog.cc:458] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [sys.catalog]: This master's current role is: LEADER
W20260808 07:05:02.080727 30260 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:05:02.080881 30260 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:05:02.080950 30261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:05:02.081267 30261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:05:02.083226 30261 catalog_manager.cc:1347] Generated new cluster ID: e647aadba5e64a9f9dc75e7f2038e67a
Aug 08 07:05:02 dist-test-slave-8tdl krb5kdc[30154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:05:02.083289 30261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:05:02.087533 30244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:02.075282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45970 (local address 127.13.207.62:35651)
0808 07:05:02.075516 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:02.075527 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:02.075579 (+ 52us) server_negotiation.cc:407] Connection header received
0808 07:05:02.076045 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:02.076053 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:02.076124 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:02.076292 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:05:02.077314 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.078849 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.079886 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.080565 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.080682 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:02.083840 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:05:02.083844 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:05:02.083845 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:05:02.083864 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:05:02.085683 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:02.086140 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:02.086144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:02.086145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:02.086264 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:02.086509 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:02.086510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:02.086511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:02.086850 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:05:02.087046 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:02.087123 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 07:05:02.087283 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":112,"threads_started":1}
I20260808 07:05:02.095187 30261 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:05:02.095809 30261 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:05:02.109004 30261 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9: Generated new TSK 0
I20260808 07:05:02.109292 30261 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
I20260808 07:05:02.164801 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35651
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:05:02.290766 30266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:05:02.291072 30266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:05:02.291123 30266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:05:02.292564 30266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:05:02.292672 30266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:05:02.292724 30266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:05:02.294540 30266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:05:02.294936 30266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:05:02.295233 30266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:05:02.301515 30271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:05:02.301438 30272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:05:02.301538 30275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:05:02.301832 30266 server_base.cc:1034] running on GCE node
I20260808 07:05:02.302076 30266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:05:02.302361 30266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:05:02.303592 30266 hybrid_clock.cc:648] HybridClock initialized: now 1786172702303585 us; error 44 us; skew 500 ppm
Aug 08 07:05:02 dist-test-slave-8tdl krb5kdc[30154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172702, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:05:02.307114 30266 init.cc:377] Logged in from keytab as kudu/127.13.207.1@KRBTEST.COM (short username kudu)
I20260808 07:05:02.307924 30266 webserver.cc:460] Webserver started at http://127.13.207.1:38031/ using document root <none> and password file <none>
I20260808 07:05:02.308460 30266 fs_manager.cc:359] Metadata directory not provided
I20260808 07:05:02.308562 30266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:05:02.308722 30266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:05:02.309897 30266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "10bff4202980420b9b474ee77febc8cd"
format_stamp: "Formatted at 2026-08-08 07:05:02 on dist-test-slave-8tdl"
server_key: "42cfc41116c6995314caf6de0a6cfabb"
server_key_iv: "d38cf94ce18fd102483d2ba7200a73c4"
server_key_version: "encryptionkey@0"
I20260808 07:05:02.310287 30266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "10bff4202980420b9b474ee77febc8cd"
format_stamp: "Formatted at 2026-08-08 07:05:02 on dist-test-slave-8tdl"
server_key: "42cfc41116c6995314caf6de0a6cfabb"
server_key_iv: "d38cf94ce18fd102483d2ba7200a73c4"
server_key_version: "encryptionkey@0"
I20260808 07:05:02.311932 30266 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:05:02.313100 30284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:02.313555 30266 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:05:02.313731 30266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "10bff4202980420b9b474ee77febc8cd"
format_stamp: "Formatted at 2026-08-08 07:05:02 on dist-test-slave-8tdl"
server_key: "42cfc41116c6995314caf6de0a6cfabb"
server_key_iv: "d38cf94ce18fd102483d2ba7200a73c4"
server_key_version: "encryptionkey@0"
I20260808 07:05:02.313830 30266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:05:02.329474 30266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:05:02.332238 30266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:05:02.332472 30266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:05:02.332785 30266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:05:02.333156 30266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:05:02.333221 30266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:02.333269 30266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:05:02.333302 30266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:02.340941 30266 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:46629
I20260808 07:05:02.340996 30406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:46629 every 8 connection(s)
I20260808 07:05:02.341392 30266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:05:02.343937 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 30266
I20260808 07:05:02.344064 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:05:02.344265 14140 external_mini_cluster.cc:1428] Setting key 68e5ee3b3cecb3793ee0dcf42046d091
Aug 08 07:05:02 dist-test-slave-8tdl krb5kdc[30154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172702, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:05:02.354741 30244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:02.342375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:60175 (local address 127.13.207.62:35651)
0808 07:05:02.342509 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:02.342511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:02.342761 (+ 250us) server_negotiation.cc:407] Connection header received
0808 07:05:02.343260 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:02.343262 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:02.343279 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:02.343362 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:05:02.344736 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.345379 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.347034 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.347274 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.347351 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:02.350735 (+ 3384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:05:02.350738 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:05:02.350738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:05:02.350753 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:05:02.352465 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:02.353265 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:02.353267 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:02.353267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:02.353343 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:02.353835 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:02.353836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:02.353837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:02.353997 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:05:02.354175 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:02.354485 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 07:05:02.354600 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260808 07:05:02.354983 30409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:02.342328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.1:60175)
0808 07:05:02.342561 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:02.342612 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 07:05:02.343150 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:05:02.343432 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:05:02.343435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:05:02.344340 (+ 905us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:05:02.344614 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:02.344619 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.345479 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.345480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:02.346886 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:02.346891 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.347369 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.347371 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:02.347546 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:02.348544 (+ 998us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:05:02.348564 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:05:02.350551 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:05:02.352587 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:05:02.352601 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:05:02.352604 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:05:02.353127 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:05:02.353592 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:05:02.353592 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:05:02.353593 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:05:02.353750 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:05:02.354154 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:05:02.354162 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:05:02.354331 (+ 169us) client_negotiation.cc:769] Sending connection context
0808 07:05:02.354450 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 07:05:02.354662 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":149,"threads_started":1}
WARNING: no policy specified for kudu/127.13.207.2@KRBTEST.COM; defaulting to no policy
I20260808 07:05:02.355909 30407 heartbeater.cc:347] Connected to a master server at 127.13.207.62:35651
I20260808 07:05:02.356022 30407 heartbeater.cc:464] Registering TS with master...
I20260808 07:05:02.356207 30407 heartbeater.cc:511] Master 127.13.207.62:35651 requested a full tablet report, sending...
I20260808 07:05:02.356841 30201 ts_manager.cc:194] Registered new tserver with Master: 10bff4202980420b9b474ee77febc8cd (127.13.207.1:46629)
I20260808 07:05:02.357606 30201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:60175
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
I20260808 07:05:02.390298 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35651
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:05:02.486625 30412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:05:02.486900 30412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:05:02.486929 30412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:05:02.488276 30412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:05:02.488327 30412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:05:02.488356 30412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:05:02.490098 30412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:05:02.490336 30412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:05:02.490550 30412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:05:02.497525 30417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:05:02.497512 30418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:05:02.497493 30420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:05:02.497776 30412 server_base.cc:1034] running on GCE node
I20260808 07:05:02.498083 30412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:05:02.498405 30412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:05:02.499594 30412 hybrid_clock.cc:648] HybridClock initialized: now 1786172702499600 us; error 51 us; skew 500 ppm
Aug 08 07:05:02 dist-test-slave-8tdl krb5kdc[30154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172702, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:05:02.502835 30412 init.cc:377] Logged in from keytab as kudu/127.13.207.2@KRBTEST.COM (short username kudu)
I20260808 07:05:02.503419 30412 webserver.cc:460] Webserver started at http://127.13.207.2:44397/ using document root <none> and password file <none>
I20260808 07:05:02.503772 30412 fs_manager.cc:359] Metadata directory not provided
I20260808 07:05:02.503849 30412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:05:02.504002 30412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:05:02.505204 30412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "eae488a44a4a4e109c01f36910a08257"
format_stamp: "Formatted at 2026-08-08 07:05:02 on dist-test-slave-8tdl"
server_key: "28be96b5057274d680cb8fc7b0755406"
server_key_iv: "0056b888050b61d69e5d77742a72d99c"
server_key_version: "encryptionkey@0"
I20260808 07:05:02.505757 30412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "eae488a44a4a4e109c01f36910a08257"
format_stamp: "Formatted at 2026-08-08 07:05:02 on dist-test-slave-8tdl"
server_key: "28be96b5057274d680cb8fc7b0755406"
server_key_iv: "0056b888050b61d69e5d77742a72d99c"
server_key_version: "encryptionkey@0"
I20260808 07:05:02.507622 30412 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 07:05:02.508615 30427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:02.509024 30412 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:05:02.509152 30412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "eae488a44a4a4e109c01f36910a08257"
format_stamp: "Formatted at 2026-08-08 07:05:02 on dist-test-slave-8tdl"
server_key: "28be96b5057274d680cb8fc7b0755406"
server_key_iv: "0056b888050b61d69e5d77742a72d99c"
server_key_version: "encryptionkey@0"
I20260808 07:05:02.509245 30412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:05:02.528878 30412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:05:02.532696 30412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:05:02.532920 30412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:05:02.533286 30412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:05:02.533653 30412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:05:02.533710 30412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:02.533753 30412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:05:02.533782 30412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:02.541028 30412 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:38893
I20260808 07:05:02.541075 30546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:38893 every 8 connection(s)
I20260808 07:05:02.541457 30412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:05:02.549189 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 30412
Aug 08 07:05:02 dist-test-slave-8tdl krb5kdc[30154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172702, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:05:02.549336 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:05:02.549543 14140 external_mini_cluster.cc:1428] Setting key 0294bc9f2f585efcaae1a5ed9a5f7e2c
I20260808 07:05:02.554309 30244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:02.542334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:45431 (local address 127.13.207.62:35651)
0808 07:05:02.542511 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:02.542513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:02.542700 (+ 187us) server_negotiation.cc:407] Connection header received
0808 07:05:02.543313 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:02.543314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:02.543328 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:02.543374 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:05:02.544473 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.545012 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.546667 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.547019 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.547151 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:02.550240 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:05:02.550242 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:05:02.550242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:05:02.550251 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:05:02.552223 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:02.552965 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:02.552968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:02.552968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:02.553056 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:02.553405 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:02.553406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:02.553406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:02.553581 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:05:02.553662 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:02.554035 (+ 373us) server_negotiation.cc:299] Negotiation successful
0808 07:05:02.554161 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260808 07:05:02.554371 30549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:02.542225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.2:45431)
0808 07:05:02.542534 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:02.542589 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 07:05:02.543189 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:05:02.543459 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:05:02.543464 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:05:02.544101 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:05:02.544353 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:02.544359 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.545147 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.545148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:02.546514 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:02.546518 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.547184 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.547186 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:02.547345 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:02.548089 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:05:02.548100 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:05:02.550087 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:05:02.552340 (+ 2253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:05:02.552357 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:05:02.552360 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:05:02.552802 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:05:02.553161 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:05:02.553162 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:05:02.553163 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:05:02.553328 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:05:02.553715 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:05:02.553723 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:05:02.553873 (+ 150us) client_negotiation.cc:769] Sending connection context
0808 07:05:02.553989 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 07:05:02.554170 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":120,"threads_started":1}
I20260808 07:05:02.555739 30547 heartbeater.cc:347] Connected to a master server at 127.13.207.62:35651
I20260808 07:05:02.555858 30547 heartbeater.cc:464] Registering TS with master...
I20260808 07:05:02.556100 30547 heartbeater.cc:511] Master 127.13.207.62:35651 requested a full tablet report, sending...
I20260808 07:05:02.556644 30201 ts_manager.cc:194] Registered new tserver with Master: eae488a44a4a4e109c01f36910a08257 (127.13.207.2:38893)
I20260808 07:05:02.557320 30201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:45431
WARNING: no policy specified for kudu/127.13.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
I20260808 07:05:02.598026 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35651
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:05:02.698819 30552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:05:02.699127 30552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:05:02.699183 30552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:05:02.700690 30552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:05:02.700778 30552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:05:02.700830 30552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:05:02.703044 30552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:05:02.703403 30552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:05:02.703701 30552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:05:02.709781 30560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:05:02.710112 30558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:05:02.709996 30552 server_base.cc:1034] running on GCE node
W20260808 07:05:02.709781 30557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:05:02.710732 30552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:05:02.711037 30552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:05:02.712352 30552 hybrid_clock.cc:648] HybridClock initialized: now 1786172702712345 us; error 42 us; skew 500 ppm
Aug 08 07:05:02 dist-test-slave-8tdl krb5kdc[30154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172702, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:05:02.715265 30552 init.cc:377] Logged in from keytab as kudu/127.13.207.3@KRBTEST.COM (short username kudu)
I20260808 07:05:02.715783 30552 webserver.cc:460] Webserver started at http://127.13.207.3:35529/ using document root <none> and password file <none>
I20260808 07:05:02.716003 30552 fs_manager.cc:359] Metadata directory not provided
I20260808 07:05:02.716048 30552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:05:02.716144 30552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:05:02.717271 30552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "c703c80db1e6405699d427084f25a688"
format_stamp: "Formatted at 2026-08-08 07:05:02 on dist-test-slave-8tdl"
server_key: "1e59e54e9ac1625e401fb40013d797c5"
server_key_iv: "3714763128bf8b0a954362e795f4c400"
server_key_version: "encryptionkey@0"
I20260808 07:05:02.717684 30552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "c703c80db1e6405699d427084f25a688"
format_stamp: "Formatted at 2026-08-08 07:05:02 on dist-test-slave-8tdl"
server_key: "1e59e54e9ac1625e401fb40013d797c5"
server_key_iv: "3714763128bf8b0a954362e795f4c400"
server_key_version: "encryptionkey@0"
I20260808 07:05:02.719641 30552 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 07:05:02.720595 30567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:02.720852 30552 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:05:02.720954 30552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "c703c80db1e6405699d427084f25a688"
format_stamp: "Formatted at 2026-08-08 07:05:02 on dist-test-slave-8tdl"
server_key: "1e59e54e9ac1625e401fb40013d797c5"
server_key_iv: "3714763128bf8b0a954362e795f4c400"
server_key_version: "encryptionkey@0"
I20260808 07:05:02.721019 30552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:05:02.733855 30552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:05:02.737381 30552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:05:02.737536 30552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:05:02.737972 30552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:05:02.738411 30552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:05:02.738478 30552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:02.738525 30552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:05:02.738557 30552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:02.746148 30552 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:37235
I20260808 07:05:02.746191 30688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:37235 every 8 connection(s)
I20260808 07:05:02.746620 30552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:05:02 dist-test-slave-8tdl krb5kdc[30154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172702, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:05:02.756193 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 30552
I20260808 07:05:02.756312 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:05:02.756636 14140 external_mini_cluster.cc:1428] Setting key 3473cf64b0eb48746a359e2a39fdbdef
I20260808 07:05:02.761881 30244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:02.748052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:47861 (local address 127.13.207.62:35651)
0808 07:05:02.748140 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:02.748143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:02.748458 (+ 315us) server_negotiation.cc:407] Connection header received
0808 07:05:02.749051 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:02.749052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:02.749064 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:02.749132 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:05:02.750369 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.751008 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.752617 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.752821 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.752895 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:02.756704 (+ 3809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:05:02.756707 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:05:02.756708 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:05:02.756722 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:05:02.759507 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:02.760131 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:02.760132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:02.760133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:02.760220 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:02.760944 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:02.760946 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:02.760947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:02.761144 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:05:02.761291 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:02.761708 (+ 417us) server_negotiation.cc:299] Negotiation successful
0808 07:05:02.761805 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 07:05:02.762172 30692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:02.747989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.3:47861)
0808 07:05:02.748282 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:02.748325 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:05:02.748878 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:05:02.749275 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:05:02.749279 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:05:02.749974 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:05:02.750216 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:02.750222 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.751243 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.751245 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:02.752392 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:02.752455 (+ 63us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.753039 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.753041 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:02.753395 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:02.754099 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:05:02.754110 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:05:02.756363 (+ 2253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:05:02.759655 (+ 3292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:05:02.759668 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:05:02.759671 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:05:02.760012 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:05:02.760394 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:05:02.760396 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:05:02.760396 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:05:02.760816 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:05:02.761314 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:05:02.761321 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:05:02.761505 (+ 184us) client_negotiation.cc:769] Sending connection context
0808 07:05:02.761625 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 07:05:02.761787 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":117,"threads_started":1}
I20260808 07:05:02.762899 30689 heartbeater.cc:347] Connected to a master server at 127.13.207.62:35651
I20260808 07:05:02.763025 30689 heartbeater.cc:464] Registering TS with master...
I20260808 07:05:02.763243 30689 heartbeater.cc:511] Master 127.13.207.62:35651 requested a full tablet report, sending...
I20260808 07:05:02.763775 30201 ts_manager.cc:194] Registered new tserver with Master: c703c80db1e6405699d427084f25a688 (127.13.207.3:37235)
I20260808 07:05:02.764202 30201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:47861
I20260808 07:05:02.771068 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:05:02.782503 30244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:02.773037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45984 (local address 127.13.207.62:35651)
0808 07:05:02.773151 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:02.773154 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:02.773174 (+ 20us) server_negotiation.cc:407] Connection header received
0808 07:05:02.773347 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:02.773348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:02.773360 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:02.773421 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:05:02.774338 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.774940 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.775919 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.776161 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.776228 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:02.778275 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:05:02.778277 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:05:02.778278 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:05:02.778287 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:05:02.780404 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:02.781230 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:02.781232 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:02.781232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:02.781309 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:02.781745 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:02.781746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:02.781746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:02.781946 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:05:02.782056 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:02.782255 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 07:05:02.782341 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 07:05:02.784665 30201 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:05:02.786381 30201 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:05:02.794883 30704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:02.790648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35354 (local address 127.13.207.3:37235)
0808 07:05:02.790907 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:02.790913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:02.791031 (+ 118us) server_negotiation.cc:407] Connection header received
0808 07:05:02.791080 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:02.791082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:02.791103 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:02.791197 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:05:02.791563 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.792499 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.793804 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.794343 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.794442 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:02.794511 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:02.794717 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 07:05:02.794769 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":132,"threads_started":1}
I20260808 07:05:02.794875 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:02.790460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:37235 (local address 127.0.0.1:35354)
0808 07:05:02.790942 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:02.790953 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:05:02.791014 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:05:02.791335 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:05:02.791338 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:05:02.791341 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:05:02.791446 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:02.791449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.792609 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.792610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:02.793707 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:02.793710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.794444 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.794445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:02.794557 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:02.794577 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 07:05:02.794703 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 07:05:02.794769 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":80,"threads_started":1}
I20260808 07:05:02.795995 30619 tablet_service.cc:1467] Processing CreateTablet for tablet 6f71623185274553b21fa427f8fba189 (DEFAULT_TABLE table=test-table [id=9516f69af1e94245873a4bb0d7e53d52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:05:02.796301 30707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:02.791077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:38893 (local address 127.0.0.1:44994)
0808 07:05:02.791452 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:02.791464 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:05:02.791519 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:05:02.791742 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:05:02.791743 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:05:02.791745 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:05:02.791791 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:02.791792 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.793202 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.793203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:02.794836 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:02.794838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.795884 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.795885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:02.796053 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:02.796065 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 07:05:02.796139 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 07:05:02.796207 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":88,"thread_start_us":84,"threads_started":1}
I20260808 07:05:02.796314 30706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:02.790886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44994 (local address 127.13.207.2:38893)
0808 07:05:02.791254 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:02.791258 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:02.791588 (+ 330us) server_negotiation.cc:407] Connection header received
0808 07:05:02.791637 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:02.791639 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:02.791658 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:02.791749 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:05:02.791861 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.793080 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.794938 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.795723 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.795797 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:02.795847 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:02.796184 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 07:05:02.796236 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":109,"threads_started":1}
I20260808 07:05:02.796398 30619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f71623185274553b21fa427f8fba189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:05:02.796427 30705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:02.790697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55264 (local address 127.13.207.1:46629)
0808 07:05:02.790937 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:02.790942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:02.790955 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:05:02.791114 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:02.791117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:02.791137 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:02.791336 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:05:02.791699 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.793032 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.794958 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.795723 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.795777 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:02.795836 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:02.796266 (+ 430us) server_negotiation.cc:299] Negotiation successful
0808 07:05:02.796314 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":95,"threads_started":1}
I20260808 07:05:02.796602 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:02.790421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:46629 (local address 127.0.0.1:55264)
0808 07:05:02.790717 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:02.790764 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 07:05:02.790833 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:05:02.791417 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:05:02.791418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:05:02.791421 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:05:02.791545 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:02.791547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.793187 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.793189 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:02.794808 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:02.794812 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.795897 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.795897 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:02.796053 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:02.796067 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 07:05:02.796140 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 07:05:02.796210 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":154,"thread_start_us":115,"threads_started":1}
I20260808 07:05:02.799695 30336 tablet_service.cc:1467] Processing CreateTablet for tablet 6f71623185274553b21fa427f8fba189 (DEFAULT_TABLE table=test-table [id=9516f69af1e94245873a4bb0d7e53d52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:05:02.799686 30478 tablet_service.cc:1467] Processing CreateTablet for tablet 6f71623185274553b21fa427f8fba189 (DEFAULT_TABLE table=test-table [id=9516f69af1e94245873a4bb0d7e53d52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:05:02.800110 30478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f71623185274553b21fa427f8fba189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:05:02.800084 30708 tablet_bootstrap.cc:492] T 6f71623185274553b21fa427f8fba189 P c703c80db1e6405699d427084f25a688: Bootstrap starting.
I20260808 07:05:02.800124 30336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f71623185274553b21fa427f8fba189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:05:02.800870 30708 tablet_bootstrap.cc:654] T 6f71623185274553b21fa427f8fba189 P c703c80db1e6405699d427084f25a688: Neither blocks nor log segments found. Creating new log.
I20260808 07:05:02.801664 30708 log.cc:824] T 6f71623185274553b21fa427f8fba189 P c703c80db1e6405699d427084f25a688: Log is configured to *not* fsync() on all Append() calls
I20260808 07:05:02.802619 30708 tablet_bootstrap.cc:492] T 6f71623185274553b21fa427f8fba189 P c703c80db1e6405699d427084f25a688: No bootstrap required, opened a new log
I20260808 07:05:02.802732 30708 ts_tablet_manager.cc:1397] T 6f71623185274553b21fa427f8fba189 P c703c80db1e6405699d427084f25a688: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:05:02.803424 30711 tablet_bootstrap.cc:492] T 6f71623185274553b21fa427f8fba189 P eae488a44a4a4e109c01f36910a08257: Bootstrap starting.
I20260808 07:05:02.804286 30712 tablet_bootstrap.cc:492] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd: Bootstrap starting.
I20260808 07:05:02.804607 30711 tablet_bootstrap.cc:654] T 6f71623185274553b21fa427f8fba189 P eae488a44a4a4e109c01f36910a08257: Neither blocks nor log segments found. Creating new log.
I20260808 07:05:02.804960 30712 tablet_bootstrap.cc:654] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd: Neither blocks nor log segments found. Creating new log.
I20260808 07:05:02.804849 30708 raft_consensus.cc:348] T 6f71623185274553b21fa427f8fba189 P c703c80db1e6405699d427084f25a688 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } }
I20260808 07:05:02.805048 30708 raft_consensus.cc:374] T 6f71623185274553b21fa427f8fba189 P c703c80db1e6405699d427084f25a688 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:05:02.805132 30708 raft_consensus.cc:729] T 6f71623185274553b21fa427f8fba189 P c703c80db1e6405699d427084f25a688 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c703c80db1e6405699d427084f25a688, State: Initialized, Role: FOLLOWER
I20260808 07:05:02.805250 30711 log.cc:824] T 6f71623185274553b21fa427f8fba189 P eae488a44a4a4e109c01f36910a08257: Log is configured to *not* fsync() on all Append() calls
I20260808 07:05:02.805294 30708 consensus_queue.cc:260] T 6f71623185274553b21fa427f8fba189 P c703c80db1e6405699d427084f25a688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } }
I20260808 07:05:02.805583 30712 log.cc:824] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd: Log is configured to *not* fsync() on all Append() calls
I20260808 07:05:02.805645 30708 ts_tablet_manager.cc:1428] T 6f71623185274553b21fa427f8fba189 P c703c80db1e6405699d427084f25a688: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:05:02.805802 30689 heartbeater.cc:503] Master 127.13.207.62:35651 was elected leader, sending a full tablet report...
I20260808 07:05:02.806396 30712 tablet_bootstrap.cc:492] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd: No bootstrap required, opened a new log
I20260808 07:05:02.806457 30711 tablet_bootstrap.cc:492] T 6f71623185274553b21fa427f8fba189 P eae488a44a4a4e109c01f36910a08257: No bootstrap required, opened a new log
I20260808 07:05:02.806468 30712 ts_tablet_manager.cc:1397] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:05:02.806527 30711 ts_tablet_manager.cc:1397] T 6f71623185274553b21fa427f8fba189 P eae488a44a4a4e109c01f36910a08257: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:05:02.808175 30712 raft_consensus.cc:348] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } }
I20260808 07:05:02.808332 30712 raft_consensus.cc:374] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:05:02.808355 30712 raft_consensus.cc:729] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10bff4202980420b9b474ee77febc8cd, State: Initialized, Role: FOLLOWER
I20260808 07:05:02.808483 30712 consensus_queue.cc:260] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } }
I20260808 07:05:02.808964 30712 ts_tablet_manager.cc:1428] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:05:02.809005 30407 heartbeater.cc:503] Master 127.13.207.62:35651 was elected leader, sending a full tablet report...
I20260808 07:05:02.809074 30711 raft_consensus.cc:348] T 6f71623185274553b21fa427f8fba189 P eae488a44a4a4e109c01f36910a08257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } }
I20260808 07:05:02.809268 30711 raft_consensus.cc:374] T 6f71623185274553b21fa427f8fba189 P eae488a44a4a4e109c01f36910a08257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:05:02.809307 30711 raft_consensus.cc:729] T 6f71623185274553b21fa427f8fba189 P eae488a44a4a4e109c01f36910a08257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eae488a44a4a4e109c01f36910a08257, State: Initialized, Role: FOLLOWER
I20260808 07:05:02.809418 30711 consensus_queue.cc:260] T 6f71623185274553b21fa427f8fba189 P eae488a44a4a4e109c01f36910a08257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } }
I20260808 07:05:02.809780 30711 ts_tablet_manager.cc:1428] T 6f71623185274553b21fa427f8fba189 P eae488a44a4a4e109c01f36910a08257: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 07:05:02.810041 30547 heartbeater.cc:503] Master 127.13.207.62:35651 was elected leader, sending a full tablet report...
W20260808 07:05:02.842480 30408 tablet.cc:2369] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:05:02.915199 30716 raft_consensus.cc:482] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:05:02.915411 30716 raft_consensus.cc:504] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } }
I20260808 07:05:02.915885 30716 leader_election.cc:288] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c703c80db1e6405699d427084f25a688 (127.13.207.3:37235), eae488a44a4a4e109c01f36910a08257 (127.13.207.2:38893)
I20260808 07:05:02.919879 30718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:02.916102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:38893 (local address 127.13.207.1:59473)
0808 07:05:02.916248 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:02.916261 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:05:02.916336 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:05:02.916631 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:05:02.916632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:05:02.916635 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:05:02.916756 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:02.916759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.917614 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.917615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:02.918625 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:02.918628 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.919496 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.919497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:02.919635 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:02.919642 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 07:05:02.919711 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 07:05:02.919771 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":63,"thread_start_us":96,"threads_started":1}
I20260808 07:05:02.920010 30706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:02.916236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:59473 (local address 127.13.207.2:38893)
0808 07:05:02.916291 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:02.916293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:02.916315 (+ 22us) server_negotiation.cc:407] Connection header received
0808 07:05:02.916512 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:02.916514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:02.916528 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:02.916602 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:05:02.916846 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.917515 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.918918 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.919379 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.919458 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:02.919470 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:02.919771 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 07:05:02.919830 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:05:02.920236 30704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:02.916442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:48945 (local address 127.13.207.3:37235)
0808 07:05:02.916508 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:02.916510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:02.916518 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:05:02.916556 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:02.916557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:02.916565 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:02.916652 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:05:02.916895 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.917519 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.918857 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.919614 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.919718 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:02.919727 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:02.920066 (+ 339us) server_negotiation.cc:299] Negotiation successful
0808 07:05:02.920131 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 07:05:02.920112 30719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:02.916185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:37235 (local address 127.13.207.1:48945)
0808 07:05:02.916366 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:02.916378 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:05:02.916453 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:05:02.916672 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:05:02.916680 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:05:02.916682 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:05:02.916782 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:02.916784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.917624 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.917625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:02.918620 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:02.918623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.919775 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.919777 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:02.919869 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:02.919874 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:05:02.919953 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 07:05:02.920003 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":38,"thread_start_us":69,"threads_started":1}
I20260808 07:05:02.920840 30500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f71623185274553b21fa427f8fba189" candidate_uuid: "10bff4202980420b9b474ee77febc8cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae488a44a4a4e109c01f36910a08257" is_pre_election: true
I20260808 07:05:02.921051 30500 raft_consensus.cc:2454] T 6f71623185274553b21fa427f8fba189 P eae488a44a4a4e109c01f36910a08257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10bff4202980420b9b474ee77febc8cd in term 0.
I20260808 07:05:02.921332 30641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f71623185274553b21fa427f8fba189" candidate_uuid: "10bff4202980420b9b474ee77febc8cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c703c80db1e6405699d427084f25a688" is_pre_election: true
I20260808 07:05:02.921429 30286 leader_election.cc:302] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10bff4202980420b9b474ee77febc8cd, eae488a44a4a4e109c01f36910a08257; no voters:
I20260808 07:05:02.921475 30641 raft_consensus.cc:2454] T 6f71623185274553b21fa427f8fba189 P c703c80db1e6405699d427084f25a688 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10bff4202980420b9b474ee77febc8cd in term 0.
I20260808 07:05:02.921566 30716 raft_consensus.cc:2781] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:05:02.921622 30716 raft_consensus.cc:482] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:05:02.921648 30716 raft_consensus.cc:3037] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:05:02.922469 30716 raft_consensus.cc:504] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } }
I20260808 07:05:02.922627 30716 leader_election.cc:288] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 1 election: Requested vote from peers c703c80db1e6405699d427084f25a688 (127.13.207.3:37235), eae488a44a4a4e109c01f36910a08257 (127.13.207.2:38893)
I20260808 07:05:02.922937 30641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f71623185274553b21fa427f8fba189" candidate_uuid: "10bff4202980420b9b474ee77febc8cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c703c80db1e6405699d427084f25a688"
I20260808 07:05:02.923005 30500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f71623185274553b21fa427f8fba189" candidate_uuid: "10bff4202980420b9b474ee77febc8cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae488a44a4a4e109c01f36910a08257"
I20260808 07:05:02.923090 30641 raft_consensus.cc:3037] T 6f71623185274553b21fa427f8fba189 P c703c80db1e6405699d427084f25a688 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:05:02.923188 30500 raft_consensus.cc:3037] T 6f71623185274553b21fa427f8fba189 P eae488a44a4a4e109c01f36910a08257 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:05:02.923995 30641 raft_consensus.cc:2454] T 6f71623185274553b21fa427f8fba189 P c703c80db1e6405699d427084f25a688 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10bff4202980420b9b474ee77febc8cd in term 1.
I20260808 07:05:02.924119 30500 raft_consensus.cc:2454] T 6f71623185274553b21fa427f8fba189 P eae488a44a4a4e109c01f36910a08257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10bff4202980420b9b474ee77febc8cd in term 1.
I20260808 07:05:02.924252 30288 leader_election.cc:302] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10bff4202980420b9b474ee77febc8cd, c703c80db1e6405699d427084f25a688; no voters:
I20260808 07:05:02.924384 30716 raft_consensus.cc:2781] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:05:02.924664 30716 raft_consensus.cc:686] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [term 1 LEADER]: Becoming Leader. State: Replica: 10bff4202980420b9b474ee77febc8cd, State: Running, Role: LEADER
I20260808 07:05:02.924803 30716 consensus_queue.cc:237] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } }
I20260808 07:05:02.926448 30201 catalog_manager.cc:5495] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 10bff4202980420b9b474ee77febc8cd (127.13.207.1). New cstate: current_term: 1 leader_uuid: "10bff4202980420b9b474ee77febc8cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } health_report { overall_health: UNKNOWN } } }
I20260808 07:05:02.945205 30705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:02.942323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55270 (local address 127.13.207.1:46629)
0808 07:05:02.942393 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:02.942395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:02.942404 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:05:02.942448 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:02.942449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:02.942469 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:02.942536 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:05:02.942711 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.943227 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.944071 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:02.944275 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:02.944335 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:02.944559 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:05:02.944683 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:05:02.944771 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:02.944925 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 07:05:02.944953 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:05:02.947528 30641 raft_consensus.cc:1261] T 6f71623185274553b21fa427f8fba189 P c703c80db1e6405699d427084f25a688 [term 1 FOLLOWER]: Refusing update from remote peer 10bff4202980420b9b474ee77febc8cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:05:02.947543 30500 raft_consensus.cc:1261] T 6f71623185274553b21fa427f8fba189 P eae488a44a4a4e109c01f36910a08257 [term 1 FOLLOWER]: Refusing update from remote peer 10bff4202980420b9b474ee77febc8cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:05:02.947980 30716 consensus_queue.cc:1035] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:05:02.948140 30720 consensus_queue.cc:1035] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:05:02.951638 30725 mvcc.cc:205] Tried to move back new op lower bound from 7316163391269892096 to 7316163391181459456. Current Snapshot: MvccSnapshot[applied={T|T < 7316163391269892096}]
I20260808 07:05:02.951668 30724 mvcc.cc:205] Tried to move back new op lower bound from 7316163391269892096 to 7316163391181459456. Current Snapshot: MvccSnapshot[applied={T|T < 7316163391269892096}]
I20260808 07:05:02.952395 30723 mvcc.cc:205] Tried to move back new op lower bound from 7316163391269892096 to 7316163391181459456. Current Snapshot: MvccSnapshot[applied={T|T < 7316163391269892096 or (T in {7316163391269892096})}]
I20260808 07:05:02.953699 30201 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:05:02.953827 30201 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:05:02.955034 30201 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9: Sending DeleteTablet for 3 replicas of tablet 6f71623185274553b21fa427f8fba189
I20260808 07:05:02.955493 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 30266
I20260808 07:05:02.955611 30619 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f71623185274553b21fa427f8fba189 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:05:02 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:35354
I20260808 07:05:02.956451 30478 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f71623185274553b21fa427f8fba189 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:05:02 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:44994
I20260808 07:05:02.957595 30733 tablet_replica.cc:330] T 6f71623185274553b21fa427f8fba189 P c703c80db1e6405699d427084f25a688: stopping tablet replica
I20260808 07:05:02.957772 30733 raft_consensus.cc:2229] T 6f71623185274553b21fa427f8fba189 P c703c80db1e6405699d427084f25a688 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:05:02.957926 30733 raft_consensus.cc:2258] T 6f71623185274553b21fa427f8fba189 P c703c80db1e6405699d427084f25a688 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:05:02.958139 30734 tablet_replica.cc:330] T 6f71623185274553b21fa427f8fba189 P eae488a44a4a4e109c01f36910a08257: stopping tablet replica
I20260808 07:05:02.958273 30734 raft_consensus.cc:2229] T 6f71623185274553b21fa427f8fba189 P eae488a44a4a4e109c01f36910a08257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:05:02.958392 30734 raft_consensus.cc:2258] T 6f71623185274553b21fa427f8fba189 P eae488a44a4a4e109c01f36910a08257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:05:02.959146 30734 ts_tablet_manager.cc:1905] T 6f71623185274553b21fa427f8fba189 P eae488a44a4a4e109c01f36910a08257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:05:02.961603 30734 ts_tablet_manager.cc:1918] T 6f71623185274553b21fa427f8fba189 P eae488a44a4a4e109c01f36910a08257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:05:02.961755 30734 log.cc:1196] T 6f71623185274553b21fa427f8fba189 P eae488a44a4a4e109c01f36910a08257: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/wal/wals/6f71623185274553b21fa427f8fba189
I20260808 07:05:02.962399 30733 ts_tablet_manager.cc:1905] T 6f71623185274553b21fa427f8fba189 P c703c80db1e6405699d427084f25a688: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:05:02.962606 30734 ts_tablet_manager.cc:1939] T 6f71623185274553b21fa427f8fba189 P eae488a44a4a4e109c01f36910a08257: Deleting consensus metadata
W20260808 07:05:02.963989 30185 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.207.1:46629 (error 108)
I20260808 07:05:02.964583 30185 catalog_manager.cc:4841] TS eae488a44a4a4e109c01f36910a08257 (127.13.207.2:38893): tablet 6f71623185274553b21fa427f8fba189 (table test-table [id=9516f69af1e94245873a4bb0d7e53d52]) successfully deleted
I20260808 07:05:02.964746 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:02.964527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:46629 (local address 127.0.0.1:55278)
0808 07:05:02.964607 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:02.964674 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.1:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 07:05:02.964879 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 30412
W20260808 07:05:02.964876 30185 catalog_manager.cc:4584] TS 10bff4202980420b9b474ee77febc8cd (127.13.207.1:46629): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6f71623185274553b21fa427f8fba189: Network error: Client connection negotiation failed: client connection to 127.13.207.1:46629: connect: Connection refused (error 111)
I20260808 07:05:02.971264 30733 ts_tablet_manager.cc:1918] T 6f71623185274553b21fa427f8fba189 P c703c80db1e6405699d427084f25a688: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:05:02.971592 30733 log.cc:1196] T 6f71623185274553b21fa427f8fba189 P c703c80db1e6405699d427084f25a688: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/6f71623185274553b21fa427f8fba189
I20260808 07:05:02.972242 30733 ts_tablet_manager.cc:1939] T 6f71623185274553b21fa427f8fba189 P c703c80db1e6405699d427084f25a688: Deleting consensus metadata
I20260808 07:05:02.973150 30186 catalog_manager.cc:4841] TS c703c80db1e6405699d427084f25a688 (127.13.207.3:37235): tablet 6f71623185274553b21fa427f8fba189 (table test-table [id=9516f69af1e94245873a4bb0d7e53d52]) successfully deleted
I20260808 07:05:02.974186 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 30552
I20260808 07:05:02.980465 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 30168
I20260808 07:05:22.989856 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:35651
--webserver_interface=127.13.207.62
--webserver_port=45219
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:35651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:05:23.081501 30736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:05:23.081822 30736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:05:23.081871 30736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:05:23.083310 30736 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 07:05:23.083388 30736 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 07:05:23.083410 30736 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 07:05:23.083431 30736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:05:23.083456 30736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:05:23.083504 30736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:05:23.083534 30736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:05:23.083557 30736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:05:23.085263 30736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:35651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:35651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=45219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:05:23.085611 30736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:05:23.085883 30736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:05:23.096496 30742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:05:23.096521 30743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:05:23.096874 30736 server_base.cc:1034] running on GCE node
W20260808 07:05:23.096496 30745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:05:23.097292 30736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:05:23.097610 30736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:05:23.098762 30736 hybrid_clock.cc:648] HybridClock initialized: now 1786172723098716 us; error 49 us; skew 500 ppm
Aug 08 07:05:23 dist-test-slave-8tdl krb5kdc[30154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172723, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:05:23.107725 30736 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:05:23.108292 30736 webserver.cc:460] Webserver started at http://127.13.207.62:45219/ using document root <none> and password file <none>
I20260808 07:05:23.108564 30736 fs_manager.cc:359] Metadata directory not provided
I20260808 07:05:23.108654 30736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:05:23.110278 30736 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:05:23.111292 30752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:23.111644 30736 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:05:23.111791 30736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "b383a3ee06cc4e09bd96131a55c522e9"
format_stamp: "Formatted at 2026-08-08 07:05:02 on dist-test-slave-8tdl"
server_key: "a44622e8bfe451ff2733e7c559433814"
server_key_iv: "4bc7fdaad653a1950f550c11d9a5f642"
server_key_version: "encryptionkey@0"
I20260808 07:05:23.112285 30736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:05:23.138106 30736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:05:23.141883 30736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:05:23.142030 30736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:05:23.146687 30736 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:35651
I20260808 07:05:23.146762 30808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:35651 every 8 connection(s)
I20260808 07:05:23.147228 30736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:05:23.147379 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 30736
I20260808 07:05:23.147706 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:46629
--local_ip_for_outbound_sockets=127.13.207.1
--tserver_master_addrs=127.13.207.62:35651
--webserver_port=38031
--webserver_interface=127.13.207.1
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:05:23.149108 30809 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:05:23.150290 30809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9: Bootstrap starting.
I20260808 07:05:23.164280 30809 log.cc:824] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9: Log is configured to *not* fsync() on all Append() calls
I20260808 07:05:23.167541 30809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:05:23.167949 30809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9: Bootstrap complete.
I20260808 07:05:23.172580 30809 raft_consensus.cc:348] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b383a3ee06cc4e09bd96131a55c522e9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35651 } }
I20260808 07:05:23.173013 30809 raft_consensus.cc:729] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b383a3ee06cc4e09bd96131a55c522e9, State: Initialized, Role: FOLLOWER
I20260808 07:05:23.173218 30809 consensus_queue.cc:260] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b383a3ee06cc4e09bd96131a55c522e9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35651 } }
I20260808 07:05:23.173295 30809 raft_consensus.cc:388] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:05:23.173355 30809 raft_consensus.cc:482] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:05:23.173429 30809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:05:23.175057 30809 raft_consensus.cc:504] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b383a3ee06cc4e09bd96131a55c522e9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35651 } }
I20260808 07:05:23.175186 30809 leader_election.cc:302] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b383a3ee06cc4e09bd96131a55c522e9; no voters:
I20260808 07:05:23.175410 30809 leader_election.cc:288] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 07:05:23.175593 30814 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 07:05:23.175681 30809 sys_catalog.cc:564] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:05:23.175994 30814 raft_consensus.cc:686] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 2 LEADER]: Becoming Leader. State: Replica: b383a3ee06cc4e09bd96131a55c522e9, State: Running, Role: LEADER
I20260808 07:05:23.176183 30814 consensus_queue.cc:237] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b383a3ee06cc4e09bd96131a55c522e9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35651 } }
I20260808 07:05:23.176918 30815 sys_catalog.cc:455] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b383a3ee06cc4e09bd96131a55c522e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b383a3ee06cc4e09bd96131a55c522e9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35651 } } }
I20260808 07:05:23.177027 30815 sys_catalog.cc:458] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:05:23.177261 30816 sys_catalog.cc:455] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b383a3ee06cc4e09bd96131a55c522e9. Latest consensus state: current_term: 2 leader_uuid: "b383a3ee06cc4e09bd96131a55c522e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b383a3ee06cc4e09bd96131a55c522e9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35651 } } }
I20260808 07:05:23.177320 30816 sys_catalog.cc:458] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:05:23.177927 30830 catalog_manager.cc:1259] Loaded cluster ID: e647aadba5e64a9f9dc75e7f2038e67a
I20260808 07:05:23.178012 30830 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9: loading cluster ID for follower catalog manager: success
I20260808 07:05:23.178975 30830 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9: acquiring CA information for follower catalog manager: success
I20260808 07:05:23.179351 30830 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:05:23.179458 30831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:05:23.179816 30831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:05:23.179925 30831 catalog_manager.cc:1259] Loaded cluster ID: e647aadba5e64a9f9dc75e7f2038e67a
I20260808 07:05:23.179983 30831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:05:23.180150 30831 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:05:23.180202 30831 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9: Loaded TSK: 0
I20260808 07:05:23.189186 30831 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9: Generated new TSK 1
I20260808 07:05:23.189368 30831 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:05:23.264243 30812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:05:23.264493 30812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:05:23.264523 30812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:05:23.265995 30812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:05:23.266052 30812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:05:23.266083 30812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:05:23.267848 30812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:46629
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=38031
--tserver_master_addrs=127.13.207.62:35651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:05:23.268133 30812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:05:23.268441 30812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:05:23.274796 30839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:05:23.274852 30843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:05:23.274966 30840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:05:23.275440 30812 server_base.cc:1034] running on GCE node
I20260808 07:05:23.275779 30812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:05:23.276185 30812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:05:23.277418 30812 hybrid_clock.cc:648] HybridClock initialized: now 1786172723277364 us; error 36 us; skew 500 ppm
Aug 08 07:05:23 dist-test-slave-8tdl krb5kdc[30154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172723, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:05:23.280596 30812 init.cc:377] Logged in from keytab as kudu/127.13.207.1@KRBTEST.COM (short username kudu)
I20260808 07:05:23.281211 30812 webserver.cc:460] Webserver started at http://127.13.207.1:38031/ using document root <none> and password file <none>
I20260808 07:05:23.281512 30812 fs_manager.cc:359] Metadata directory not provided
I20260808 07:05:23.281610 30812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:05:23.283490 30812 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:05:23.284390 30850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:23.284606 30812 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:05:23.284708 30812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "10bff4202980420b9b474ee77febc8cd"
format_stamp: "Formatted at 2026-08-08 07:05:02 on dist-test-slave-8tdl"
server_key: "42cfc41116c6995314caf6de0a6cfabb"
server_key_iv: "d38cf94ce18fd102483d2ba7200a73c4"
server_key_version: "encryptionkey@0"
I20260808 07:05:23.285142 30812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:05:23.299116 30812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:05:23.301625 30812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:05:23.301810 30812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:05:23.302237 30812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:05:23.302987 30857 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 07:05:23.304581 30812 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 07:05:23.304643 30812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 07:05:23.304672 30812 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 07:05:23.305364 30812 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 07:05:23.305411 30812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 07:05:23.305464 30857 tablet_bootstrap.cc:492] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd: Bootstrap starting.
I20260808 07:05:23.314239 30812 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:46629
I20260808 07:05:23.314574 30970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:46629 every 8 connection(s)
I20260808 07:05:23.314893 30812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:05:23.316355 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 30812
I20260808 07:05:23.316705 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:38893
--local_ip_for_outbound_sockets=127.13.207.2
--tserver_master_addrs=127.13.207.62:35651
--webserver_port=44397
--webserver_interface=127.13.207.2
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:05:23.319438 30857 log.cc:824] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd: Log is configured to *not* fsync() on all Append() calls
I20260808 07:05:23.324088 30857 tablet_bootstrap.cc:492] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:05:23.324474 30857 tablet_bootstrap.cc:492] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd: Bootstrap complete.
I20260808 07:05:23.326167 30857 ts_tablet_manager.cc:1397] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd: Time spent bootstrapping tablet: real 0.021s user 0.017s sys 0.000s
Aug 08 07:05:23 dist-test-slave-8tdl krb5kdc[30154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172723, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:05:23.328270 30857 raft_consensus.cc:348] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } }
I20260808 07:05:23.328717 30857 raft_consensus.cc:729] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10bff4202980420b9b474ee77febc8cd, State: Initialized, Role: FOLLOWER
I20260808 07:05:23.328941 30857 consensus_queue.cc:260] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } }
I20260808 07:05:23.329310 30857 ts_tablet_manager.cc:1428] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
I20260808 07:05:23.334831 30974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:23.315826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.1:34235)
0808 07:05:23.318852 (+ 3026us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:23.318931 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0808 07:05:23.319670 (+ 739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:05:23.320479 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:05:23.320484 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:05:23.320942 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:05:23.321366 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:23.321375 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.323530 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.323531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:23.324981 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:23.324986 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.326118 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.326119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:23.326286 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:23.327054 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:05:23.327064 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:05:23.329317 (+ 2253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:05:23.332193 (+ 2876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:05:23.332203 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:05:23.332205 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:05:23.332550 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:05:23.332877 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:05:23.332877 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:05:23.332878 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:05:23.333036 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:05:23.333845 (+ 809us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:05:23.333853 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:05:23.334156 (+ 303us) client_negotiation.cc:769] Sending connection context
0808 07:05:23.334292 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 07:05:23.334540 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2965,"thread_start_us":103,"threads_started":1}
I20260808 07:05:23.335300 30975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:23.315925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:34235 (local address 127.13.207.62:35651)
0808 07:05:23.319910 (+ 3985us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:23.319920 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:23.319966 (+ 46us) server_negotiation.cc:407] Connection header received
0808 07:05:23.320188 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:23.320195 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:23.320258 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:23.321528 (+ 1270us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:05:23.321627 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.323327 (+ 1700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.325148 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.325948 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.326643 (+ 695us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:23.329554 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:05:23.329560 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:05:23.329560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:05:23.329576 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:05:23.332053 (+ 2477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:23.332658 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:23.332662 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:23.332662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:23.332800 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:23.333113 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:23.333114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:23.333114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:23.333707 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:05:23.333804 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:23.334891 (+ 1087us) server_negotiation.cc:299] Negotiation successful
0808 07:05:23.335069 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3927,"thread_start_us":86,"threads_started":1}
I20260808 07:05:23.335986 30971 heartbeater.cc:347] Connected to a master server at 127.13.207.62:35651
I20260808 07:05:23.336086 30971 heartbeater.cc:464] Registering TS with master...
I20260808 07:05:23.336274 30971 heartbeater.cc:511] Master 127.13.207.62:35651 requested a full tablet report, sending...
I20260808 07:05:23.337100 30770 ts_manager.cc:194] Registered new tserver with Master: 10bff4202980420b9b474ee77febc8cd (127.13.207.1:46629)
I20260808 07:05:23.338410 30770 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:34235
I20260808 07:05:23.339558 30971 heartbeater.cc:503] Master 127.13.207.62:35651 was elected leader, sending a full tablet report...
Aug 08 07:05:23 dist-test-slave-8tdl krb5kdc[30154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172723, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for kudu/127.13.207.1@KRBTEST.COM
I20260808 07:05:23.349028 30979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:23.337815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:46629 (local address 127.0.0.1:38814)
0808 07:05:23.338381 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:23.338425 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:05:23.338502 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:05:23.338928 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:05:23.338934 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:05:23.339323 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:05:23.339450 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:23.339452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.340942 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.340944 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:23.341914 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:23.341918 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.342649 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.342650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:23.342772 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:23.343448 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:05:23.343454 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:05:23.345266 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:05:23.347651 (+ 2385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:05:23.347660 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:05:23.347663 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:05:23.347937 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:05:23.348196 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:05:23.348196 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:05:23.348197 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:05:23.348265 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:05:23.348611 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:05:23.348619 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:05:23.348680 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 07:05:23.348765 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 07:05:23.348904 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":125,"threads_started":1}
I20260808 07:05:23.349035 30981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:23.338125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38814 (local address 127.13.207.1:46629)
0808 07:05:23.338657 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:23.338662 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:23.338759 (+ 97us) server_negotiation.cc:407] Connection header received
0808 07:05:23.338801 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:23.338804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:23.338820 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:23.339544 (+ 724us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:05:23.339607 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.340829 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.342012 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.342560 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.342638 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:23.345398 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:05:23.345410 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:05:23.345410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:05:23.345420 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:05:23.347548 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:23.348034 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:23.348037 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:23.348037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:23.348116 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:23.348334 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:23.348334 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:23.348335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:23.348538 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:05:23.348629 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:23.348821 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 07:05:23.348926 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":78,"threads_started":1}
I20260808 07:05:23.349974 30900 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f71623185274553b21fa427f8fba189 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:05:02 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:38814
I20260808 07:05:23.350747 30982 tablet_replica.cc:330] stopping tablet replica
I20260808 07:05:23.350864 30982 raft_consensus.cc:2229] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:05:23.350936 30982 raft_consensus.cc:2258] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:05:23.351271 30982 ts_tablet_manager.cc:1905] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:05:23.353996 30982 ts_tablet_manager.cc:1918] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:05:23.354099 30982 log.cc:1196] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/wal/wals/6f71623185274553b21fa427f8fba189
I20260808 07:05:23.354353 30982 ts_tablet_manager.cc:1939] T 6f71623185274553b21fa427f8fba189 P 10bff4202980420b9b474ee77febc8cd: Deleting consensus metadata
I20260808 07:05:23.355072 30754 catalog_manager.cc:4841] TS 10bff4202980420b9b474ee77febc8cd (127.13.207.1:46629): tablet 6f71623185274553b21fa427f8fba189 (table test-table [id=9516f69af1e94245873a4bb0d7e53d52]) successfully deleted
W20260808 07:05:23.421442 30976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:05:23.422008 30976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:05:23.422083 30976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:05:23.424678 30976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:05:23.424773 30976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:05:23.424814 30976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:05:23.427124 30976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:38893
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=44397
--tserver_master_addrs=127.13.207.62:35651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:05:23.427443 30976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:05:23.427805 30976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:05:23.434520 30987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:05:23.434527 30988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:05:23.434558 30990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:05:23.434885 30976 server_base.cc:1034] running on GCE node
I20260808 07:05:23.435169 30976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:05:23.435446 30976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:05:23.436617 30976 hybrid_clock.cc:648] HybridClock initialized: now 1786172723436593 us; error 42 us; skew 500 ppm
Aug 08 07:05:23 dist-test-slave-8tdl krb5kdc[30154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172723, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:05:23.439594 30976 init.cc:377] Logged in from keytab as kudu/127.13.207.2@KRBTEST.COM (short username kudu)
I20260808 07:05:23.440125 30976 webserver.cc:460] Webserver started at http://127.13.207.2:44397/ using document root <none> and password file <none>
I20260808 07:05:23.440384 30976 fs_manager.cc:359] Metadata directory not provided
I20260808 07:05:23.440466 30976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:05:23.442040 30976 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:05:23.442830 30997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:23.443230 30976 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:05:23.443352 30976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "eae488a44a4a4e109c01f36910a08257"
format_stamp: "Formatted at 2026-08-08 07:05:02 on dist-test-slave-8tdl"
server_key: "28be96b5057274d680cb8fc7b0755406"
server_key_iv: "0056b888050b61d69e5d77742a72d99c"
server_key_version: "encryptionkey@0"
I20260808 07:05:23.443758 30976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:05:23.453246 30976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:05:23.456012 30976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:05:23.456252 30976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:05:23.456571 30976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:05:23.456955 30976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:05:23.456995 30976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:23.457042 30976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:05:23.457082 30976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:23.464067 30976 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:38893
I20260808 07:05:23.464154 31116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:38893 every 8 connection(s)
I20260808 07:05:23.464578 30976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:05:23.464985 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 30976
I20260808 07:05:23.465333 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:37235
--local_ip_for_outbound_sockets=127.13.207.3
--tserver_master_addrs=127.13.207.62:35651
--webserver_port=35529
--webserver_interface=127.13.207.3
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:05:23 dist-test-slave-8tdl krb5kdc[30154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172723, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:05:23.476799 30975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:23.465784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:43713 (local address 127.13.207.62:35651)
0808 07:05:23.465836 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:23.465838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:23.466205 (+ 367us) server_negotiation.cc:407] Connection header received
0808 07:05:23.466864 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:23.466866 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:23.466886 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:23.466934 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:05:23.467980 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.468529 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.469627 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.469921 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.470006 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:23.473162 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:05:23.473164 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:05:23.473164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:05:23.473175 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:05:23.474975 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:23.475564 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:23.475566 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:23.475567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:23.475634 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:23.475933 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:23.475934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:23.475934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:23.476087 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:05:23.476150 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:23.476581 (+ 431us) server_negotiation.cc:299] Negotiation successful
0808 07:05:23.476694 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:05:23.477083 31119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:23.465783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.2:43713)
0808 07:05:23.466005 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:23.466103 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0808 07:05:23.466734 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:05:23.467020 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:05:23.467024 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:05:23.467650 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:05:23.467863 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:23.467868 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.468647 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.468648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:23.469510 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:23.469513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.470021 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.470022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:23.470168 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:23.470934 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:05:23.470944 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:05:23.472954 (+ 2010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:05:23.475082 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:05:23.475094 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:05:23.475096 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:05:23.475470 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:05:23.475703 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:05:23.475704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:05:23.475704 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:05:23.475852 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:05:23.476151 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:05:23.476160 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:05:23.476369 (+ 209us) client_negotiation.cc:769] Sending connection context
0808 07:05:23.476543 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 07:05:23.476754 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":115,"threads_started":1}
I20260808 07:05:23.477929 31117 heartbeater.cc:347] Connected to a master server at 127.13.207.62:35651
I20260808 07:05:23.478065 31117 heartbeater.cc:464] Registering TS with master...
I20260808 07:05:23.478271 31117 heartbeater.cc:511] Master 127.13.207.62:35651 requested a full tablet report, sending...
I20260808 07:05:23.479100 30770 ts_manager.cc:194] Registered new tserver with Master: eae488a44a4a4e109c01f36910a08257 (127.13.207.2:38893)
I20260808 07:05:23.479702 30770 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:43713
W20260808 07:05:23.563490 31120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:05:23.563747 31120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:05:23.563776 31120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:05:23.565026 31120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:05:23.565073 31120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:05:23.565104 31120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:05:23.567085 31120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:37235
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=35529
--tserver_master_addrs=127.13.207.62:35651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:05:23.567365 31120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:05:23.567603 31120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:05:23.574061 31129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:05:23.574074 31127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:05:23.574384 31120 server_base.cc:1034] running on GCE node
W20260808 07:05:23.574093 31126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:05:23.574909 31120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:05:23.575189 31120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:05:23.576400 31120 hybrid_clock.cc:648] HybridClock initialized: now 1786172723576380 us; error 37 us; skew 500 ppm
Aug 08 07:05:23 dist-test-slave-8tdl krb5kdc[30154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172723, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:05:23.579373 31120 init.cc:377] Logged in from keytab as kudu/127.13.207.3@KRBTEST.COM (short username kudu)
I20260808 07:05:23.579965 31120 webserver.cc:460] Webserver started at http://127.13.207.3:35529/ using document root <none> and password file <none>
I20260808 07:05:23.580224 31120 fs_manager.cc:359] Metadata directory not provided
I20260808 07:05:23.580319 31120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:05:23.581804 31120 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:05:23.582655 31136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:23.582952 31120 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:05:23.583052 31120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "c703c80db1e6405699d427084f25a688"
format_stamp: "Formatted at 2026-08-08 07:05:02 on dist-test-slave-8tdl"
server_key: "1e59e54e9ac1625e401fb40013d797c5"
server_key_iv: "3714763128bf8b0a954362e795f4c400"
server_key_version: "encryptionkey@0"
I20260808 07:05:23.583459 31120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:05:23.601711 31120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:05:23.604491 31120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:05:23.604640 31120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:05:23.604874 31120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:05:23.605247 31120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:05:23.605288 31120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:23.605314 31120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:05:23.605329 31120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:23.612425 31120 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:37235
I20260808 07:05:23.612489 31259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:37235 every 8 connection(s)
I20260808 07:05:23.612887 31120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:05:23 dist-test-slave-8tdl krb5kdc[30154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172723, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:05:23.622766 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 31120
I20260808 07:05:23.624145 30975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:23.613721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:49839 (local address 127.13.207.62:35651)
0808 07:05:23.613777 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:23.613779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:23.614094 (+ 315us) server_negotiation.cc:407] Connection header received
0808 07:05:23.614533 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:23.614535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:23.614549 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:23.614621 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:05:23.615837 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.616443 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.617671 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.617879 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.617941 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:23.620490 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:05:23.620492 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:05:23.620493 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:05:23.620501 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:05:23.622202 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:23.622811 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:23.622812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:23.622813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:23.622893 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:23.623340 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:23.623341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:23.623342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:23.623494 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:05:23.623580 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:23.623935 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 07:05:23.624042 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:05:23.624375 31262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:23.613687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.3:49839)
0808 07:05:23.613872 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:23.613917 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:05:23.614426 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:05:23.614741 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:05:23.614745 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:05:23.615516 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:05:23.615727 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:23.615732 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.616648 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.616650 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:23.617579 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:23.617582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.617964 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.617965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:23.618124 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:23.618725 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:05:23.618736 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:05:23.620357 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:05:23.622312 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:05:23.622322 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:05:23.622325 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:05:23.622662 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:05:23.623052 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:05:23.623054 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:05:23.623054 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:05:23.623241 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:05:23.623590 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:05:23.623600 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:05:23.623769 (+ 169us) client_negotiation.cc:769] Sending connection context
0808 07:05:23.623893 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 07:05:23.624094 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":101,"threads_started":1}
I20260808 07:05:23.624935 31260 heartbeater.cc:347] Connected to a master server at 127.13.207.62:35651
I20260808 07:05:23.625082 31260 heartbeater.cc:464] Registering TS with master...
I20260808 07:05:23.625311 31260 heartbeater.cc:511] Master 127.13.207.62:35651 requested a full tablet report, sending...
I20260808 07:05:23.625741 30770 ts_manager.cc:194] Registered new tserver with Master: c703c80db1e6405699d427084f25a688 (127.13.207.3:37235)
I20260808 07:05:23.626116 30770 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:49839
I20260808 07:05:23.630970 31265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:23.623217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58852 (local address 127.13.207.62:35651)
0808 07:05:23.623379 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:23.623381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:23.623499 (+ 118us) server_negotiation.cc:407] Connection header received
0808 07:05:23.623571 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:23.623572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:23.623585 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:23.623676 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:05:23.624635 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.625313 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.626381 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.626730 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.626811 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:23.628153 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:05:23.628154 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:05:23.628155 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:05:23.628162 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:05:23.629615 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:23.630069 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:23.630070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:23.630070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:23.630128 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:23.630369 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:23.630370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:23.630370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:23.630484 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:05:23.630650 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:23.630742 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 07:05:23.630830 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":91,"threads_started":1}
I20260808 07:05:23.631855 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:05:23.634670 31265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:23.632295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58864 (local address 127.13.207.62:35651)
0808 07:05:23.632333 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:23.632334 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:23.632528 (+ 194us) server_negotiation.cc:407] Connection header received
0808 07:05:23.632567 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:23.632567 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:23.632599 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:23.632659 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:05:23.632782 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.633291 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.634019 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.634233 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.634293 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:23.634438 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:05:23.634492 (+ 54us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 07:05:23.634619 (+ 127us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58864: token expired
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 07:05:23.634825 31265 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58864: token expired
W20260808 07:05:23.635746 30693 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.13.207.62:35651)
I20260808 07:05:23.642890 31265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:23.636078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58866 (local address 127.13.207.62:35651)
0808 07:05:23.636152 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:23.636154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:23.636164 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:05:23.636254 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:23.636255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:23.636266 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:23.636337 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:05:23.636925 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.637436 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.638258 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.638465 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.638558 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:23.639968 (+ 1410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:05:23.639970 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:05:23.639970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:05:23.640048 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:05:23.641582 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:23.641997 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:23.641998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:23.641999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:23.642065 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:23.642297 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:23.642298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:23.642299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:23.642424 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:05:23.642545 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:23.642660 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 07:05:23.642776 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:05:23.644423 30770 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:05:23.646620 30770 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:05:23.649516 30900 tablet_service.cc:1467] Processing CreateTablet for tablet 3724adb84ef34a50bf06642ee2ba03c5 (DEFAULT_TABLE table=test-table [id=3d315b6457324f6796c756513fe4029f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:05:23.649799 30900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3724adb84ef34a50bf06642ee2ba03c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:05:23.651387 30857 tablet_bootstrap.cc:492] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd: Bootstrap starting.
I20260808 07:05:23.651775 30857 tablet_bootstrap.cc:654] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd: Neither blocks nor log segments found. Creating new log.
I20260808 07:05:23.652411 30857 tablet_bootstrap.cc:492] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd: No bootstrap required, opened a new log
I20260808 07:05:23.652498 30857 ts_tablet_manager.cc:1397] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:05:23.652832 30857 raft_consensus.cc:348] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } }
I20260808 07:05:23.652999 30857 raft_consensus.cc:374] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:05:23.653066 30857 raft_consensus.cc:729] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10bff4202980420b9b474ee77febc8cd, State: Initialized, Role: FOLLOWER
I20260808 07:05:23.653116 30857 consensus_queue.cc:260] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } }
I20260808 07:05:23.653208 30857 ts_tablet_manager.cc:1428] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:05:23.653915 30979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:23.648958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:37235 (local address 127.0.0.1:55736)
0808 07:05:23.649053 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:23.649065 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:05:23.649121 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:05:23.649744 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:05:23.649745 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:05:23.649748 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:05:23.649802 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:23.649804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.651244 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.651246 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:23.652466 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:23.652470 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.653631 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.653632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:23.653712 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:23.653742 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 07:05:23.653774 (+ 32us) client_negotiation.cc:240] Negotiation successful
0808 07:05:23.653826 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260808 07:05:23.654356 31269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:23.649277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55736 (local address 127.13.207.3:37235)
0808 07:05:23.649519 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:23.649523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:23.649533 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:05:23.649575 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:23.649576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:23.649609 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:23.649701 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:05:23.649918 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.651137 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.652617 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.653528 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.654112 (+ 584us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:23.654196 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:23.654225 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 07:05:23.654272 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":148,"threads_started":1}
I20260808 07:05:23.654939 31267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:23.649053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:38893 (local address 127.0.0.1:56462)
0808 07:05:23.649525 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:23.649538 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:05:23.649605 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:05:23.649913 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:05:23.649914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:05:23.649916 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:05:23.649964 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:23.649965 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.652595 (+ 2630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.652596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:23.653646 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:23.653648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.654629 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.654630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:23.654734 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:23.654739 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:05:23.654799 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 07:05:23.654859 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":69,"threads_started":1}
I20260808 07:05:23.654954 31268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:23.649195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56462 (local address 127.13.207.2:38893)
0808 07:05:23.649396 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:23.649401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:23.649612 (+ 211us) server_negotiation.cc:407] Connection header received
0808 07:05:23.649705 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:23.649707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:23.649724 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:23.649796 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:05:23.650051 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.651148 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.653745 (+ 2597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.654535 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.654628 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:23.654682 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:23.654831 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 07:05:23.654875 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":117,"threads_started":1}
I20260808 07:05:23.655223 31189 tablet_service.cc:1467] Processing CreateTablet for tablet 3724adb84ef34a50bf06642ee2ba03c5 (DEFAULT_TABLE table=test-table [id=3d315b6457324f6796c756513fe4029f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:05:23.655601 31189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3724adb84ef34a50bf06642ee2ba03c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:05:23.655570 31046 tablet_service.cc:1467] Processing CreateTablet for tablet 3724adb84ef34a50bf06642ee2ba03c5 (DEFAULT_TABLE table=test-table [id=3d315b6457324f6796c756513fe4029f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:05:23.655911 31046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3724adb84ef34a50bf06642ee2ba03c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:05:23.659037 31272 tablet_bootstrap.cc:492] T 3724adb84ef34a50bf06642ee2ba03c5 P c703c80db1e6405699d427084f25a688: Bootstrap starting.
I20260808 07:05:23.659771 31272 tablet_bootstrap.cc:654] T 3724adb84ef34a50bf06642ee2ba03c5 P c703c80db1e6405699d427084f25a688: Neither blocks nor log segments found. Creating new log.
I20260808 07:05:23.660336 31272 log.cc:824] T 3724adb84ef34a50bf06642ee2ba03c5 P c703c80db1e6405699d427084f25a688: Log is configured to *not* fsync() on all Append() calls
I20260808 07:05:23.660643 31271 tablet_bootstrap.cc:492] T 3724adb84ef34a50bf06642ee2ba03c5 P eae488a44a4a4e109c01f36910a08257: Bootstrap starting.
I20260808 07:05:23.661263 31271 tablet_bootstrap.cc:654] T 3724adb84ef34a50bf06642ee2ba03c5 P eae488a44a4a4e109c01f36910a08257: Neither blocks nor log segments found. Creating new log.
I20260808 07:05:23.661367 31272 tablet_bootstrap.cc:492] T 3724adb84ef34a50bf06642ee2ba03c5 P c703c80db1e6405699d427084f25a688: No bootstrap required, opened a new log
I20260808 07:05:23.661458 31272 ts_tablet_manager.cc:1397] T 3724adb84ef34a50bf06642ee2ba03c5 P c703c80db1e6405699d427084f25a688: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:05:23.661904 31271 log.cc:824] T 3724adb84ef34a50bf06642ee2ba03c5 P eae488a44a4a4e109c01f36910a08257: Log is configured to *not* fsync() on all Append() calls
I20260808 07:05:23.662966 31271 tablet_bootstrap.cc:492] T 3724adb84ef34a50bf06642ee2ba03c5 P eae488a44a4a4e109c01f36910a08257: No bootstrap required, opened a new log
I20260808 07:05:23.663058 31271 ts_tablet_manager.cc:1397] T 3724adb84ef34a50bf06642ee2ba03c5 P eae488a44a4a4e109c01f36910a08257: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260808 07:05:23.663556 31272 raft_consensus.cc:348] T 3724adb84ef34a50bf06642ee2ba03c5 P c703c80db1e6405699d427084f25a688 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } }
I20260808 07:05:23.663748 31272 raft_consensus.cc:374] T 3724adb84ef34a50bf06642ee2ba03c5 P c703c80db1e6405699d427084f25a688 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:05:23.663797 31272 raft_consensus.cc:729] T 3724adb84ef34a50bf06642ee2ba03c5 P c703c80db1e6405699d427084f25a688 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c703c80db1e6405699d427084f25a688, State: Initialized, Role: FOLLOWER
I20260808 07:05:23.663930 31272 consensus_queue.cc:260] T 3724adb84ef34a50bf06642ee2ba03c5 P c703c80db1e6405699d427084f25a688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } }
I20260808 07:05:23.664222 31272 ts_tablet_manager.cc:1428] T 3724adb84ef34a50bf06642ee2ba03c5 P c703c80db1e6405699d427084f25a688: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:05:23.664287 31260 heartbeater.cc:503] Master 127.13.207.62:35651 was elected leader, sending a full tablet report...
I20260808 07:05:23.665740 31271 raft_consensus.cc:348] T 3724adb84ef34a50bf06642ee2ba03c5 P eae488a44a4a4e109c01f36910a08257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } }
I20260808 07:05:23.666126 31271 raft_consensus.cc:374] T 3724adb84ef34a50bf06642ee2ba03c5 P eae488a44a4a4e109c01f36910a08257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:05:23.666193 31271 raft_consensus.cc:729] T 3724adb84ef34a50bf06642ee2ba03c5 P eae488a44a4a4e109c01f36910a08257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eae488a44a4a4e109c01f36910a08257, State: Initialized, Role: FOLLOWER
I20260808 07:05:23.666328 31271 consensus_queue.cc:260] T 3724adb84ef34a50bf06642ee2ba03c5 P eae488a44a4a4e109c01f36910a08257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } }
I20260808 07:05:23.666723 31271 ts_tablet_manager.cc:1428] T 3724adb84ef34a50bf06642ee2ba03c5 P eae488a44a4a4e109c01f36910a08257: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260808 07:05:23.666766 31117 heartbeater.cc:503] Master 127.13.207.62:35651 was elected leader, sending a full tablet report...
I20260808 07:05:23.691932 30978 raft_consensus.cc:482] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:05:23.692193 30978 raft_consensus.cc:504] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } }
I20260808 07:05:23.692607 30978 leader_election.cc:288] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eae488a44a4a4e109c01f36910a08257 (127.13.207.2:38893), c703c80db1e6405699d427084f25a688 (127.13.207.3:37235)
I20260808 07:05:23.695798 31268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:23.692982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:38385 (local address 127.13.207.2:38893)
0808 07:05:23.693050 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:23.693052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:23.693063 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:05:23.693100 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:23.693101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:23.693109 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:23.693181 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:05:23.693351 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.693835 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.694990 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.695382 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.695472 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:23.695483 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:23.695668 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 07:05:23.695715 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:05:23.695849 31269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:23.692961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:53317 (local address 127.13.207.3:37235)
0808 07:05:23.693014 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:23.693017 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:23.693189 (+ 172us) server_negotiation.cc:407] Connection header received
0808 07:05:23.693264 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:23.693265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:23.693274 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:23.693341 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:05:23.693545 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.694002 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.695068 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.695453 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.695558 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:23.695568 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:23.695730 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 07:05:23.695775 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:05:23.695863 30974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:23.692828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:38893 (local address 127.13.207.1:38385)
0808 07:05:23.692896 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:23.692911 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:05:23.692967 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:05:23.693184 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:05:23.693185 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:05:23.693187 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:05:23.693265 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:23.693267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.693935 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.693937 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:23.694877 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:23.694880 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.695469 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.695470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:23.695562 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:23.695586 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 07:05:23.695661 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 07:05:23.695709 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 07:05:23.695878 31277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:23.692831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:37235 (local address 127.13.207.1:53317)
0808 07:05:23.693102 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:23.693111 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:05:23.693205 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:05:23.693357 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:05:23.693358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:05:23.693360 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:05:23.693453 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:23.693454 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.694106 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.694108 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:23.694953 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:23.694955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.695564 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.695565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:23.695643 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:23.695647 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:05:23.695710 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 07:05:23.695752 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":20,"thread_start_us":118,"threads_started":1}
I20260808 07:05:23.696311 31067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3724adb84ef34a50bf06642ee2ba03c5" candidate_uuid: "10bff4202980420b9b474ee77febc8cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae488a44a4a4e109c01f36910a08257" is_pre_election: true
I20260808 07:05:23.696502 31067 raft_consensus.cc:2454] T 3724adb84ef34a50bf06642ee2ba03c5 P eae488a44a4a4e109c01f36910a08257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10bff4202980420b9b474ee77febc8cd in term 0.
I20260808 07:05:23.696516 31209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3724adb84ef34a50bf06642ee2ba03c5" candidate_uuid: "10bff4202980420b9b474ee77febc8cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c703c80db1e6405699d427084f25a688" is_pre_election: true
I20260808 07:05:23.696699 31209 raft_consensus.cc:2454] T 3724adb84ef34a50bf06642ee2ba03c5 P c703c80db1e6405699d427084f25a688 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10bff4202980420b9b474ee77febc8cd in term 0.
I20260808 07:05:23.696785 30852 leader_election.cc:302] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10bff4202980420b9b474ee77febc8cd, eae488a44a4a4e109c01f36910a08257; no voters:
I20260808 07:05:23.696923 30978 raft_consensus.cc:2781] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:05:23.697000 30978 raft_consensus.cc:482] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:05:23.697017 30978 raft_consensus.cc:3037] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:05:23.697804 30978 raft_consensus.cc:504] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } }
I20260808 07:05:23.697990 30978 leader_election.cc:288] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 1 election: Requested vote from peers eae488a44a4a4e109c01f36910a08257 (127.13.207.2:38893), c703c80db1e6405699d427084f25a688 (127.13.207.3:37235)
I20260808 07:05:23.698124 31067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3724adb84ef34a50bf06642ee2ba03c5" candidate_uuid: "10bff4202980420b9b474ee77febc8cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae488a44a4a4e109c01f36910a08257"
I20260808 07:05:23.698218 31067 raft_consensus.cc:3037] T 3724adb84ef34a50bf06642ee2ba03c5 P eae488a44a4a4e109c01f36910a08257 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:05:23.698210 31209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3724adb84ef34a50bf06642ee2ba03c5" candidate_uuid: "10bff4202980420b9b474ee77febc8cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c703c80db1e6405699d427084f25a688"
I20260808 07:05:23.698307 31209 raft_consensus.cc:3037] T 3724adb84ef34a50bf06642ee2ba03c5 P c703c80db1e6405699d427084f25a688 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:05:23.698997 31067 raft_consensus.cc:2454] T 3724adb84ef34a50bf06642ee2ba03c5 P eae488a44a4a4e109c01f36910a08257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10bff4202980420b9b474ee77febc8cd in term 1.
I20260808 07:05:23.699182 30852 leader_election.cc:302] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10bff4202980420b9b474ee77febc8cd, eae488a44a4a4e109c01f36910a08257; no voters:
I20260808 07:05:23.699245 31209 raft_consensus.cc:2454] T 3724adb84ef34a50bf06642ee2ba03c5 P c703c80db1e6405699d427084f25a688 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10bff4202980420b9b474ee77febc8cd in term 1.
I20260808 07:05:23.699311 30978 raft_consensus.cc:2781] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:05:23.699506 30978 raft_consensus.cc:686] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 1 LEADER]: Becoming Leader. State: Replica: 10bff4202980420b9b474ee77febc8cd, State: Running, Role: LEADER
I20260808 07:05:23.699597 30978 consensus_queue.cc:237] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } }
I20260808 07:05:23.700472 30769 catalog_manager.cc:5495] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 10bff4202980420b9b474ee77febc8cd (127.13.207.1). New cstate: current_term: 1 leader_uuid: "10bff4202980420b9b474ee77febc8cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } health_report { overall_health: HEALTHY } } }
W20260808 07:05:23.715454 31118 tablet.cc:2369] T 3724adb84ef34a50bf06642ee2ba03c5 P eae488a44a4a4e109c01f36910a08257: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:05:23.718199 30981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:23.714727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38822 (local address 127.13.207.1:46629)
0808 07:05:23.714827 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:23.714829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:23.714838 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:05:23.714908 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:23.714909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:23.714930 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:23.715089 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:05:23.715342 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.716023 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.717071 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:23.717304 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:23.717372 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:23.717601 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:05:23.717724 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:05:23.717788 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:23.717980 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 07:05:23.718094 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 07:05:23.720126 31067 raft_consensus.cc:1261] T 3724adb84ef34a50bf06642ee2ba03c5 P eae488a44a4a4e109c01f36910a08257 [term 1 FOLLOWER]: Refusing update from remote peer 10bff4202980420b9b474ee77febc8cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:05:23.720263 31209 raft_consensus.cc:1261] T 3724adb84ef34a50bf06642ee2ba03c5 P c703c80db1e6405699d427084f25a688 [term 1 FOLLOWER]: Refusing update from remote peer 10bff4202980420b9b474ee77febc8cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:05:23.720644 30978 consensus_queue.cc:1035] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:05:23.720830 31278 consensus_queue.cc:1035] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:05:23.723541 31280 mvcc.cc:205] Tried to move back new op lower bound from 7316163476354445312 to 7316163476274126848. Current Snapshot: MvccSnapshot[applied={T|T < 7316163476354445312}]
I20260808 07:05:23.725931 31281 mvcc.cc:205] Tried to move back new op lower bound from 7316163476354445312 to 7316163476274126848. Current Snapshot: MvccSnapshot[applied={T|T < 7316163476354445312}]
I20260808 07:05:23.726563 30770 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:05:23.726713 30770 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:05:23.728220 30770 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9: Sending DeleteTablet for 3 replicas of tablet 3724adb84ef34a50bf06642ee2ba03c5
I20260808 07:05:23.728685 30900 tablet_service.cc:1514] Processing DeleteTablet for tablet 3724adb84ef34a50bf06642ee2ba03c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:05:23 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:38814
I20260808 07:05:23.728674 31189 tablet_service.cc:1514] Processing DeleteTablet for tablet 3724adb84ef34a50bf06642ee2ba03c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:05:23 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:55736
I20260808 07:05:23.728894 31188 tablet_service.cc:1514] Processing DeleteTablet for tablet 3724adb84ef34a50bf06642ee2ba03c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:05:23 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:55736
I20260808 07:05:23.728986 30982 tablet_replica.cc:330] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd: stopping tablet replica
I20260808 07:05:23.729059 30982 raft_consensus.cc:2229] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:05:23.729198 30982 raft_consensus.cc:2258] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:05:23.729354 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 30812
I20260808 07:05:23.731283 31046 tablet_service.cc:1514] Processing DeleteTablet for tablet 3724adb84ef34a50bf06642ee2ba03c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:05:23 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:56462
I20260808 07:05:23.732101 31290 tablet_replica.cc:330] T 3724adb84ef34a50bf06642ee2ba03c5 P c703c80db1e6405699d427084f25a688: stopping tablet replica
I20260808 07:05:23.732263 31290 raft_consensus.cc:2229] T 3724adb84ef34a50bf06642ee2ba03c5 P c703c80db1e6405699d427084f25a688 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:05:23.732419 31290 raft_consensus.cc:2258] T 3724adb84ef34a50bf06642ee2ba03c5 P c703c80db1e6405699d427084f25a688 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:05:23.732949 31291 tablet_replica.cc:330] T 3724adb84ef34a50bf06642ee2ba03c5 P eae488a44a4a4e109c01f36910a08257: stopping tablet replica
I20260808 07:05:23.733026 31290 ts_tablet_manager.cc:1905] T 3724adb84ef34a50bf06642ee2ba03c5 P c703c80db1e6405699d427084f25a688: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:05:23.733109 31291 raft_consensus.cc:2229] T 3724adb84ef34a50bf06642ee2ba03c5 P eae488a44a4a4e109c01f36910a08257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:05:23.733217 31291 raft_consensus.cc:2258] T 3724adb84ef34a50bf06642ee2ba03c5 P eae488a44a4a4e109c01f36910a08257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:05:23.733678 31291 ts_tablet_manager.cc:1905] T 3724adb84ef34a50bf06642ee2ba03c5 P eae488a44a4a4e109c01f36910a08257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:05:23.735723 31290 ts_tablet_manager.cc:1918] T 3724adb84ef34a50bf06642ee2ba03c5 P c703c80db1e6405699d427084f25a688: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:05:23.735865 31290 log.cc:1196] T 3724adb84ef34a50bf06642ee2ba03c5 P c703c80db1e6405699d427084f25a688: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/3724adb84ef34a50bf06642ee2ba03c5
I20260808 07:05:23.736021 31291 ts_tablet_manager.cc:1918] T 3724adb84ef34a50bf06642ee2ba03c5 P eae488a44a4a4e109c01f36910a08257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:05:23.736115 31291 log.cc:1196] T 3724adb84ef34a50bf06642ee2ba03c5 P eae488a44a4a4e109c01f36910a08257: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/wal/wals/3724adb84ef34a50bf06642ee2ba03c5
I20260808 07:05:23.736284 31290 ts_tablet_manager.cc:1939] T 3724adb84ef34a50bf06642ee2ba03c5 P c703c80db1e6405699d427084f25a688: Deleting consensus metadata
I20260808 07:05:23.736687 31291 ts_tablet_manager.cc:1939] T 3724adb84ef34a50bf06642ee2ba03c5 P eae488a44a4a4e109c01f36910a08257: Deleting consensus metadata
I20260808 07:05:23.737237 30755 catalog_manager.cc:4841] TS c703c80db1e6405699d427084f25a688 (127.13.207.3:37235): tablet 3724adb84ef34a50bf06642ee2ba03c5 (table test-table [id=3d315b6457324f6796c756513fe4029f]) successfully deleted
I20260808 07:05:23.737251 30754 catalog_manager.cc:4841] TS eae488a44a4a4e109c01f36910a08257 (127.13.207.2:38893): tablet 3724adb84ef34a50bf06642ee2ba03c5 (table test-table [id=3d315b6457324f6796c756513fe4029f]) successfully deleted
W20260808 07:05:23.737449 30755 catalog_manager.cc:4808] TS c703c80db1e6405699d427084f25a688 (127.13.207.3:37235): delete failed for tablet 3724adb84ef34a50bf06642ee2ba03c5 because the tablet was not found. No further retry: Not found: Tablet not found: 3724adb84ef34a50bf06642ee2ba03c5
W20260808 07:05:23.739866 30754 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.207.1:46629 (error 108)
I20260808 07:05:23.740381 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 30976
I20260808 07:05:23.740530 31267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:23.740368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:46629 (local address 127.0.0.1:38826)
0808 07:05:23.740435 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:23.740476 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.1:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 07:05:23.746562 30754 catalog_manager.cc:4584] TS 10bff4202980420b9b474ee77febc8cd (127.13.207.1:46629): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3724adb84ef34a50bf06642ee2ba03c5: Network error: Client connection negotiation failed: client connection to 127.13.207.1:46629: connect: Connection refused (error 111)
I20260808 07:05:23.752588 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 31120
I20260808 07:05:23.762902 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 30736
I20260808 07:05:43.771858 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:35651
--webserver_interface=127.13.207.62
--webserver_port=45219
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:35651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:05:43.866456 31293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:05:43.866798 31293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:05:43.866832 31293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:05:43.868181 31293 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 07:05:43.868294 31293 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 07:05:43.868320 31293 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 07:05:43.868340 31293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:05:43.868400 31293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:05:43.868422 31293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:05:43.868439 31293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:05:43.868462 31293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:05:43.871258 31293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:35651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:35651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=45219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:05:43.871585 31293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:05:43.871907 31293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:05:43.878520 31293 server_base.cc:1034] running on GCE node
W20260808 07:05:43.878450 31300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:05:43.878737 31299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:05:43.878450 31302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:05:43.879096 31293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:05:43.879467 31293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:05:43.880702 31293 hybrid_clock.cc:648] HybridClock initialized: now 1786172743880613 us; error 102 us; skew 500 ppm
Aug 08 07:05:43 dist-test-slave-8tdl krb5kdc[30154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172743, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:05:43.891570 31293 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:05:43.892381 31293 webserver.cc:460] Webserver started at http://127.13.207.62:45219/ using document root <none> and password file <none>
I20260808 07:05:43.892717 31293 fs_manager.cc:359] Metadata directory not provided
I20260808 07:05:43.892776 31293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:05:43.896277 31293 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 07:05:43.898023 31309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:43.898564 31293 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:05:43.898741 31293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "b383a3ee06cc4e09bd96131a55c522e9"
format_stamp: "Formatted at 2026-08-08 07:05:02 on dist-test-slave-8tdl"
server_key: "a44622e8bfe451ff2733e7c559433814"
server_key_iv: "4bc7fdaad653a1950f550c11d9a5f642"
server_key_version: "encryptionkey@0"
I20260808 07:05:43.899294 31293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:05:43.941828 31293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:05:43.944454 31293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:05:43.944589 31293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:05:43.949067 31293 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:35651
I20260808 07:05:43.949198 31364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:35651 every 8 connection(s)
I20260808 07:05:43.949566 31293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:05:43.949849 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 31293
I20260808 07:05:43.950174 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:46629
--local_ip_for_outbound_sockets=127.13.207.1
--tserver_master_addrs=127.13.207.62:35651
--webserver_port=38031
--webserver_interface=127.13.207.1
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:05:43.951598 31365 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:05:43.952378 31365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9: Bootstrap starting.
I20260808 07:05:43.965919 31365 log.cc:824] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9: Log is configured to *not* fsync() on all Append() calls
I20260808 07:05:43.969576 31365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:05:43.970036 31365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9: Bootstrap complete.
I20260808 07:05:43.974354 31365 raft_consensus.cc:348] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b383a3ee06cc4e09bd96131a55c522e9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35651 } }
I20260808 07:05:43.974771 31365 raft_consensus.cc:729] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b383a3ee06cc4e09bd96131a55c522e9, State: Initialized, Role: FOLLOWER
I20260808 07:05:43.974982 31365 consensus_queue.cc:260] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b383a3ee06cc4e09bd96131a55c522e9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35651 } }
I20260808 07:05:43.975061 31365 raft_consensus.cc:388] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:05:43.975114 31365 raft_consensus.cc:482] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:05:43.975167 31365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 2 FOLLOWER]: Advancing to term 3
I20260808 07:05:43.976756 31365 raft_consensus.cc:504] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b383a3ee06cc4e09bd96131a55c522e9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35651 } }
I20260808 07:05:43.976904 31365 leader_election.cc:302] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b383a3ee06cc4e09bd96131a55c522e9; no voters:
I20260808 07:05:43.977124 31365 leader_election.cc:288] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 07:05:43.977227 31370 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 07:05:43.977413 31365 sys_catalog.cc:564] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:05:43.977442 31370 raft_consensus.cc:686] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [term 3 LEADER]: Becoming Leader. State: Replica: b383a3ee06cc4e09bd96131a55c522e9, State: Running, Role: LEADER
I20260808 07:05:43.977556 31370 consensus_queue.cc:237] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b383a3ee06cc4e09bd96131a55c522e9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35651 } }
I20260808 07:05:43.978104 31371 sys_catalog.cc:455] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b383a3ee06cc4e09bd96131a55c522e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b383a3ee06cc4e09bd96131a55c522e9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35651 } } }
I20260808 07:05:43.978188 31371 sys_catalog.cc:458] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:05:43.978353 31372 sys_catalog.cc:455] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b383a3ee06cc4e09bd96131a55c522e9. Latest consensus state: current_term: 3 leader_uuid: "b383a3ee06cc4e09bd96131a55c522e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b383a3ee06cc4e09bd96131a55c522e9" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35651 } } }
I20260808 07:05:43.978413 31372 sys_catalog.cc:458] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:05:43.978631 31379 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:05:43.979521 31379 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:05:43.979642 31379 catalog_manager.cc:1259] Loaded cluster ID: e647aadba5e64a9f9dc75e7f2038e67a
I20260808 07:05:43.979893 31379 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:05:43.980700 31379 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:05:43.980875 31379 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9: Loaded TSK: 0
I20260808 07:05:43.980942 31379 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b383a3ee06cc4e09bd96131a55c522e9: Loaded TSK: 1
F20260808 07:05:43.981172 31379 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fbc738dc9ad google::LogMessage::Fail() at ??:0
@ 0x7fbc738df88a google::LogMessage::SendToLog() at ??:0
@ 0x7fbc738dc4de google::LogMessage::Flush() at ??:0
@ 0x7fbc738dda19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fbc76049104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fbc76046d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fbc7b8361cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fbc7b83648e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fbc7b836594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fbc7b7fcc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fbc7b806ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fbc757aa0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fbc7579f954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fbc766036db start_thread at ??:0
@ 0x7fbc723da61f clone at ??:0
W20260808 07:05:44.071278 31367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:05:44.071869 31367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:05:44.071954 31367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:05:44.073647 31367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:05:44.073724 31367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:05:44.073760 31367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:05:44.076409 31367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:46629
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=38031
--tserver_master_addrs=127.13.207.62:35651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:05:44.076783 31367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:05:44.077112 31367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:05:44.083427 31409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:05:44.083527 31408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:05:44.083635 31367 server_base.cc:1034] running on GCE node
W20260808 07:05:44.083737 31412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:05:44.083985 31367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:05:44.084313 31367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:05:44.085526 31367 hybrid_clock.cc:648] HybridClock initialized: now 1786172744085469 us; error 66 us; skew 500 ppm
Aug 08 07:05:44 dist-test-slave-8tdl krb5kdc[30154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172744, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:05:44.089124 31367 init.cc:377] Logged in from keytab as kudu/127.13.207.1@KRBTEST.COM (short username kudu)
I20260808 07:05:44.089980 31367 webserver.cc:460] Webserver started at http://127.13.207.1:38031/ using document root <none> and password file <none>
I20260808 07:05:44.090372 31367 fs_manager.cc:359] Metadata directory not provided
I20260808 07:05:44.090443 31367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:05:44.092324 31367 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:05:44.093456 31421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:44.093672 31367 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260808 07:05:44.093771 31367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "10bff4202980420b9b474ee77febc8cd"
format_stamp: "Formatted at 2026-08-08 07:05:02 on dist-test-slave-8tdl"
server_key: "42cfc41116c6995314caf6de0a6cfabb"
server_key_iv: "d38cf94ce18fd102483d2ba7200a73c4"
server_key_version: "encryptionkey@0"
I20260808 07:05:44.094267 31367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:05:44.123050 31367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:05:44.126535 31367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:05:44.126770 31367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:05:44.127094 31367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:05:44.127702 31428 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 07:05:44.129936 31367 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 07:05:44.130034 31367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260808 07:05:44.130088 31367 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 07:05:44.131211 31367 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 07:05:44.131273 31367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260808 07:05:44.131394 31428 tablet_bootstrap.cc:492] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd: Bootstrap starting.
I20260808 07:05:44.142212 31367 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:46629
I20260808 07:05:44.142439 31541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:46629 every 8 connection(s)
I20260808 07:05:44.142817 31367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:05:44.145491 31428 log.cc:824] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd: Log is configured to *not* fsync() on all Append() calls
I20260808 07:05:44.148475 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 31367
I20260808 07:05:44.148736 31428 tablet_bootstrap.cc:492] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:05:44.148867 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:38893
--local_ip_for_outbound_sockets=127.13.207.2
--tserver_master_addrs=127.13.207.62:35651
--webserver_port=44397
--webserver_interface=127.13.207.2
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:05:44.149086 31428 tablet_bootstrap.cc:492] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd: Bootstrap complete.
I20260808 07:05:44.150494 31428 ts_tablet_manager.cc:1397] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.014s
I20260808 07:05:44.151898 31428 raft_consensus.cc:348] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } }
I20260808 07:05:44.152200 31428 raft_consensus.cc:729] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10bff4202980420b9b474ee77febc8cd, State: Initialized, Role: FOLLOWER
I20260808 07:05:44.152407 31428 consensus_queue.cc:260] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } }
I20260808 07:05:44.152808 31428 ts_tablet_manager.cc:1428] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd: Time spent starting tablet: real 0.002s user 0.002s sys 0.004s
W20260808 07:05:44.240636 31544 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:05:44.143679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.1:53315)
0808 07:05:44.143926 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:44.143993 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 07:05:44.145131 (+ 1138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:05:44.240423 (+ 95292us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":117,"threads_started":1}
W20260808 07:05:44.240972 31542 heartbeater.cc:644] Failed to heartbeat to 127.13.207.62:35651 (0 consecutive failures): Network error: Failed to ping master at 127.13.207.62:35651: Client connection negotiation failed: client connection to 127.13.207.62:35651: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 07:05:44.241392 31544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:44.241254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.1:46733)
0808 07:05:44.241318 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:44.241363 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 07:05:44.241787 31544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:44.241693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.1:45607)
0808 07:05:44.241736 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:44.241762 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 07:05:44.241899 31542 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:05:44.245328 31546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:05:44.245597 31546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:05:44.245648 31546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:05:44.246941 31546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:05:44.247004 31546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:05:44.247051 31546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:05:44.248631 31546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:38893
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=44397
--tserver_master_addrs=127.13.207.62:35651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:05:44.249161 31546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:05:44.249466 31546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:05:44.255844 31553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:05:44.255848 31556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:05:44.256017 31546 server_base.cc:1034] running on GCE node
W20260808 07:05:44.255836 31554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:05:44.256438 31546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:05:44.256788 31546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:05:44.258021 31546 hybrid_clock.cc:648] HybridClock initialized: now 1786172744258004 us; error 36 us; skew 500 ppm
Aug 08 07:05:44 dist-test-slave-8tdl krb5kdc[30154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172744, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:05:44.261078 31546 init.cc:377] Logged in from keytab as kudu/127.13.207.2@KRBTEST.COM (short username kudu)
I20260808 07:05:44.261602 31546 webserver.cc:460] Webserver started at http://127.13.207.2:44397/ using document root <none> and password file <none>
I20260808 07:05:44.261837 31546 fs_manager.cc:359] Metadata directory not provided
I20260808 07:05:44.261883 31546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:05:44.263458 31546 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:05:44.264261 31563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:44.264500 31546 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:05:44.264607 31546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "eae488a44a4a4e109c01f36910a08257"
format_stamp: "Formatted at 2026-08-08 07:05:02 on dist-test-slave-8tdl"
server_key: "28be96b5057274d680cb8fc7b0755406"
server_key_iv: "0056b888050b61d69e5d77742a72d99c"
server_key_version: "encryptionkey@0"
I20260808 07:05:44.265014 31546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:05:44.284122 31546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:05:44.286670 31546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:05:44.286922 31546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:05:44.287151 31546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:05:44.287514 31546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:05:44.287554 31546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:44.287577 31546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:05:44.287591 31546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:44.295006 31546 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:38893
I20260808 07:05:44.295082 31680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:38893 every 8 connection(s)
I20260808 07:05:44.295482 31546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:05:44.296762 31683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:44.296249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.2:40923)
0808 07:05:44.296452 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:44.296584 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":98,"threads_started":1}
W20260808 07:05:44.297226 31681 heartbeater.cc:644] Failed to heartbeat to 127.13.207.62:35651 (0 consecutive failures): Network error: Failed to ping master at 127.13.207.62:35651: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
I20260808 07:05:44.297536 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 31546
I20260808 07:05:44.297691 31683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:44.297505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.2:54755)
0808 07:05:44.297589 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:44.297655 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 07:05:44.297905 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:37235
--local_ip_for_outbound_sockets=127.13.207.3
--tserver_master_addrs=127.13.207.62:35651
--webserver_port=35529
--webserver_interface=127.13.207.3
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:05:44.298298 31683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:44.298139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.2:37681)
0808 07:05:44.298201 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:44.298267 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 07:05:44.298557 31681 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:05:44.386505 31684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:05:44.386868 31684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:05:44.386929 31684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:05:44.388263 31684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:05:44.388343 31684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:05:44.388392 31684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:05:44.390038 31684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:37235
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=35529
--tserver_master_addrs=127.13.207.62:35651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:05:44.390352 31684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:05:44.390609 31684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:05:44.396378 31689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:05:44.396385 31692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:05:44.396471 31690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:05:44.396783 31684 server_base.cc:1034] running on GCE node
I20260808 07:05:44.397032 31684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:05:44.397303 31684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:05:44.398535 31684 hybrid_clock.cc:648] HybridClock initialized: now 1786172744398524 us; error 39 us; skew 500 ppm
Aug 08 07:05:44 dist-test-slave-8tdl krb5kdc[30154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172744, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:05:44.401399 31684 init.cc:377] Logged in from keytab as kudu/127.13.207.3@KRBTEST.COM (short username kudu)
I20260808 07:05:44.402006 31684 webserver.cc:460] Webserver started at http://127.13.207.3:35529/ using document root <none> and password file <none>
I20260808 07:05:44.402261 31684 fs_manager.cc:359] Metadata directory not provided
I20260808 07:05:44.402333 31684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:05:44.403949 31684 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:05:44.404822 31700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:44.405228 31684 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:05:44.405366 31684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "c703c80db1e6405699d427084f25a688"
format_stamp: "Formatted at 2026-08-08 07:05:02 on dist-test-slave-8tdl"
server_key: "1e59e54e9ac1625e401fb40013d797c5"
server_key_iv: "3714763128bf8b0a954362e795f4c400"
server_key_version: "encryptionkey@0"
I20260808 07:05:44.405786 31684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:05:44.424717 31684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:05:44.427456 31684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:05:44.427814 31684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:05:44.428151 31684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:05:44.428542 31684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:05:44.428608 31684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:44.428656 31684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:05:44.428690 31684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:05:44.436096 31684 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:37235
I20260808 07:05:44.436237 31818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:37235 every 8 connection(s)
I20260808 07:05:44.436640 31684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:05:44.438014 31821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:44.437509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.3:48299)
0808 07:05:44.437702 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:44.437792 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":77,"threads_started":1}
W20260808 07:05:44.438552 31819 heartbeater.cc:644] Failed to heartbeat to 127.13.207.62:35651 (0 consecutive failures): Network error: Failed to ping master at 127.13.207.62:35651: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
I20260808 07:05:44.439064 31821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:44.438834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.3:58515)
0808 07:05:44.438991 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:44.439032 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260808 07:05:44.439572 31821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:44.439426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.3:38751)
0808 07:05:44.439516 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:44.439549 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260808 07:05:44.439769 31819 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:05:44.445293 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 31684
I20260808 07:05:45.242808 31823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:45.242420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.1:51511)
0808 07:05:45.242665 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:45.242739 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":83,"threads_started":1}
I20260808 07:05:45.299275 31824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:45.299023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.2:60469)
0808 07:05:45.299163 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:45.299210 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":63,"threads_started":1}
I20260808 07:05:45.440496 31825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:45.440188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.3:43053)
0808 07:05:45.440379 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:45.440425 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":80,"threads_started":1}
I20260808 07:05:45.667312 31826 raft_consensus.cc:482] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:05:45.667469 31826 raft_consensus.cc:504] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } }
I20260808 07:05:45.667927 31826 leader_election.cc:288] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eae488a44a4a4e109c01f36910a08257 (127.13.207.2:38893), c703c80db1e6405699d427084f25a688 (127.13.207.3:37235)
Aug 08 07:05:45 dist-test-slave-8tdl krb5kdc[30154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172744, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.2@KRBTEST.COM
Aug 08 07:05:45 dist-test-slave-8tdl krb5kdc[30154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172744, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.3@KRBTEST.COM
I20260808 07:05:45.679879 31827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:45.668195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:38893 (local address 127.13.207.1:52155)
0808 07:05:45.668406 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:45.668415 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:05:45.668485 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:05:45.669328 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:05:45.669332 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:05:45.669762 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:05:45.669961 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:45.669967 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:45.671150 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:45.671151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:45.672081 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:45.672084 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:45.672772 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:45.672773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:45.672949 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:45.673476 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:05:45.673485 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:05:45.675245 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:05:45.677541 (+ 2296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:05:45.677549 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:05:45.677551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:05:45.677858 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:05:45.678297 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:05:45.678298 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:05:45.678298 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:05:45.678502 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:05:45.679153 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:05:45.679163 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:05:45.679339 (+ 176us) client_negotiation.cc:769] Sending connection context
0808 07:05:45.679544 (+ 205us) client_negotiation.cc:240] Negotiation successful
0808 07:05:45.679751 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":184,"thread_start_us":98,"threads_started":1}
I20260808 07:05:45.679908 31829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:45.668371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:52155 (local address 127.13.207.2:38893)
0808 07:05:45.668547 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:45.668558 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:45.668596 (+ 38us) server_negotiation.cc:407] Connection header received
0808 07:05:45.669065 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:45.669067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:45.669114 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:45.669276 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:05:45.670098 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:45.671036 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:45.672189 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:45.672686 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:45.672858 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:45.675434 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:05:45.675439 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:05:45.675439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:05:45.675453 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:05:45.677403 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:45.677976 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:45.677979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:45.677979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:45.678187 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:45.678611 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:45.678613 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:45.678613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:45.679065 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:05:45.679159 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:45.679623 (+ 464us) server_negotiation.cc:299] Negotiation successful
0808 07:05:45.679784 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":74,"threads_started":1}
I20260808 07:05:45.679951 31823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:45.668186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:37235 (local address 127.13.207.1:42995)
0808 07:05:45.668227 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:45.668240 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:05:45.668299 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:05:45.669384 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:05:45.669386 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:05:45.669763 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:05:45.669961 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:45.669967 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:45.671144 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:45.671145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:45.672079 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:05:45.672082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:05:45.672751 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:05:45.672752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:05:45.672866 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:45.673541 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:05:45.673546 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:05:45.675636 (+ 2090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:05:45.677793 (+ 2157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:05:45.677794 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:05:45.677794 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:05:45.678048 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:05:45.678377 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:05:45.678378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:05:45.678378 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:05:45.678502 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:05:45.679153 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:05:45.679163 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:05:45.679339 (+ 176us) client_negotiation.cc:769] Sending connection context
0808 07:05:45.679491 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 07:05:45.679810 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":161}
I20260808 07:05:45.679914 31828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:45.668350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:42995 (local address 127.13.207.3:37235)
0808 07:05:45.668526 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 07:05:45.668540 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 07:05:45.668579 (+ 39us) server_negotiation.cc:407] Connection header received
0808 07:05:45.669175 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:05:45.669179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:05:45.669239 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:05:45.669390 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:05:45.670103 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:45.671033 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:45.672172 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:05:45.672645 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:05:45.672762 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:05:45.675773 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:05:45.675777 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:05:45.675777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:05:45.675787 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:05:45.677703 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:45.678150 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:45.678159 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:45.678159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:45.678290 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:05:45.678612 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:05:45.678613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:05:45.678613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:05:45.679018 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:05:45.679127 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 07:05:45.679624 (+ 497us) server_negotiation.cc:299] Negotiation successful
0808 07:05:45.679784 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":67,"threads_started":1}
W20260808 07:05:45.680750 31423 leader_election.cc:341] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eae488a44a4a4e109c01f36910a08257 (127.13.207.2:38893): Not found: Tablet not found: 3724adb84ef34a50bf06642ee2ba03c5
W20260808 07:05:45.681325 31424 leader_election.cc:341] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c703c80db1e6405699d427084f25a688 (127.13.207.3:37235): Not found: Tablet not found: 3724adb84ef34a50bf06642ee2ba03c5
I20260808 07:05:45.681383 31424 leader_election.cc:302] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10bff4202980420b9b474ee77febc8cd; no voters: c703c80db1e6405699d427084f25a688, eae488a44a4a4e109c01f36910a08257
I20260808 07:05:45.681500 31826 raft_consensus.cc:2726] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:05:46.243700 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:46.243433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.1:59077)
0808 07:05:46.243600 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:46.243650 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":72,"threads_started":1}
I20260808 07:05:46.300205 31831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:46.299958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.2:40657)
0808 07:05:46.300115 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:46.300160 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":79,"threads_started":1}
I20260808 07:05:46.441449 31832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:46.441155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.3:40845)
0808 07:05:46.441341 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:46.441399 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":90,"threads_started":1}
I20260808 07:05:47.244599 31833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:47.244305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.1:52931)
0808 07:05:47.244484 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:47.244547 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":67,"threads_started":1}
I20260808 07:05:47.301142 31834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:47.300863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.2:57779)
0808 07:05:47.301042 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:47.301093 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":89,"threads_started":1}
I20260808 07:05:47.442400 31835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:47.442101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.3:47649)
0808 07:05:47.442268 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:47.442313 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":74,"threads_started":1}
I20260808 07:05:47.533605 31836 raft_consensus.cc:482] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:05:47.533753 31836 raft_consensus.cc:504] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } }
I20260808 07:05:47.534070 31836 leader_election.cc:288] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eae488a44a4a4e109c01f36910a08257 (127.13.207.2:38893), c703c80db1e6405699d427084f25a688 (127.13.207.3:37235)
W20260808 07:05:47.534552 31424 leader_election.cc:341] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c703c80db1e6405699d427084f25a688 (127.13.207.3:37235): Not found: Tablet not found: 3724adb84ef34a50bf06642ee2ba03c5
W20260808 07:05:47.534642 31423 leader_election.cc:341] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eae488a44a4a4e109c01f36910a08257 (127.13.207.2:38893): Not found: Tablet not found: 3724adb84ef34a50bf06642ee2ba03c5
I20260808 07:05:47.534773 31423 leader_election.cc:302] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10bff4202980420b9b474ee77febc8cd; no voters: c703c80db1e6405699d427084f25a688, eae488a44a4a4e109c01f36910a08257
I20260808 07:05:47.534899 31836 raft_consensus.cc:2726] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:05:48.245522 31838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:48.245235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.1:46537)
0808 07:05:48.245416 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:48.245469 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":73,"threads_started":1}
I20260808 07:05:48.302073 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:48.301826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.2:35001)
0808 07:05:48.301986 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:48.302024 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
I20260808 07:05:48.443331 31840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:48.443050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.3:34033)
0808 07:05:48.443237 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:48.443281 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":98,"threads_started":1}
I20260808 07:05:49.246524 31841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:49.246255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.1:38227)
0808 07:05:49.246421 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:49.246475 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":73,"threads_started":1}
I20260808 07:05:49.303121 31842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:49.302800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.2:60927)
0808 07:05:49.303028 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:49.303068 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":117,"threads_started":1}
I20260808 07:05:49.444278 31843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:49.444000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.3:39217)
0808 07:05:49.444177 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:49.444226 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":72,"threads_started":1}
I20260808 07:05:49.525028 31844 raft_consensus.cc:482] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:05:49.525151 31844 raft_consensus.cc:504] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } }
I20260808 07:05:49.525374 31844 leader_election.cc:288] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eae488a44a4a4e109c01f36910a08257 (127.13.207.2:38893), c703c80db1e6405699d427084f25a688 (127.13.207.3:37235)
W20260808 07:05:49.526048 31424 leader_election.cc:341] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c703c80db1e6405699d427084f25a688 (127.13.207.3:37235): Not found: Tablet not found: 3724adb84ef34a50bf06642ee2ba03c5
W20260808 07:05:49.526150 31423 leader_election.cc:341] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eae488a44a4a4e109c01f36910a08257 (127.13.207.2:38893): Not found: Tablet not found: 3724adb84ef34a50bf06642ee2ba03c5
I20260808 07:05:49.526234 31423 leader_election.cc:302] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10bff4202980420b9b474ee77febc8cd; no voters: c703c80db1e6405699d427084f25a688, eae488a44a4a4e109c01f36910a08257
I20260808 07:05:49.526362 31844 raft_consensus.cc:2726] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:05:50.247293 31845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:50.247056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.1:35569)
0808 07:05:50.247202 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:50.247246 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
I20260808 07:05:50.304070 31847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:50.303792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.2:46905)
0808 07:05:50.303965 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:50.304016 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":90,"threads_started":1}
I20260808 07:05:50.445127 31848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:50.444898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.3:54597)
0808 07:05:50.445034 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:50.445078 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":71,"threads_started":1}
I20260808 07:05:51.248263 31849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:51.247908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.1:33049)
0808 07:05:51.248158 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:51.248212 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":104,"threads_started":1}
I20260808 07:05:51.304944 31850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:51.304675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.2:55387)
0808 07:05:51.304844 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:51.304892 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":70,"threads_started":1}
I20260808 07:05:51.445911 31851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:51.445645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.3:51561)
0808 07:05:51.445815 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:51.445860 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":69,"threads_started":1}
I20260808 07:05:52.249158 31852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:52.248851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.1:46179)
0808 07:05:52.249063 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:52.249109 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":124,"threads_started":1}
I20260808 07:05:52.305832 31853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:52.305524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.2:42871)
0808 07:05:52.305736 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:52.305783 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":120,"threads_started":1}
I20260808 07:05:52.446882 31854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:52.446540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.3:55435)
0808 07:05:52.446779 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:52.446829 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":74,"threads_started":1}
I20260808 07:05:53.250108 31855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:53.249843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.1:44451)
0808 07:05:53.250008 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:53.250057 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":76,"threads_started":1}
I20260808 07:05:53.306761 31856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:53.306467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.2:50299)
0808 07:05:53.306615 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:53.306662 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260808 07:05:53.447793 31857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:53.447525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.3:55045)
0808 07:05:53.447697 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:53.447742 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":91,"threads_started":1}
I20260808 07:05:53.619138 31858 raft_consensus.cc:482] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:05:53.619247 31858 raft_consensus.cc:504] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } }
I20260808 07:05:53.619607 31858 leader_election.cc:288] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eae488a44a4a4e109c01f36910a08257 (127.13.207.2:38893), c703c80db1e6405699d427084f25a688 (127.13.207.3:37235)
W20260808 07:05:53.620064 31424 leader_election.cc:341] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c703c80db1e6405699d427084f25a688 (127.13.207.3:37235): Not found: Tablet not found: 3724adb84ef34a50bf06642ee2ba03c5
W20260808 07:05:53.620127 31423 leader_election.cc:341] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eae488a44a4a4e109c01f36910a08257 (127.13.207.2:38893): Not found: Tablet not found: 3724adb84ef34a50bf06642ee2ba03c5
I20260808 07:05:53.620158 31424 leader_election.cc:302] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10bff4202980420b9b474ee77febc8cd; no voters: c703c80db1e6405699d427084f25a688, eae488a44a4a4e109c01f36910a08257
I20260808 07:05:53.620236 31858 raft_consensus.cc:2726] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:05:54.250931 31859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:54.250632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.1:37251)
0808 07:05:54.250840 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:54.250880 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":139,"threads_started":1}
I20260808 07:05:54.307587 31860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:54.307316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.2:55741)
0808 07:05:54.307481 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:54.307527 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":76,"threads_started":1}
I20260808 07:05:54.448866 31861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:54.448533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.3:40795)
0808 07:05:54.448765 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:54.448814 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":107,"threads_started":1}
I20260808 07:05:55.252013 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:55.251629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.1:37661)
0808 07:05:55.251917 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:55.251963 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":147,"threads_started":1}
I20260808 07:05:55.308528 31863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:55.308221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.2:55669)
0808 07:05:55.308414 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:55.308476 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":94,"threads_started":1}
I20260808 07:05:55.348841 31864 raft_consensus.cc:482] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:05:55.348959 31864 raft_consensus.cc:504] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae488a44a4a4e109c01f36910a08257" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 38893 } } peers { permanent_uuid: "c703c80db1e6405699d427084f25a688" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 37235 } } peers { permanent_uuid: "10bff4202980420b9b474ee77febc8cd" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 46629 } }
I20260808 07:05:55.349226 31864 leader_election.cc:288] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eae488a44a4a4e109c01f36910a08257 (127.13.207.2:38893), c703c80db1e6405699d427084f25a688 (127.13.207.3:37235)
W20260808 07:05:55.349748 31424 leader_election.cc:341] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c703c80db1e6405699d427084f25a688 (127.13.207.3:37235): Not found: Tablet not found: 3724adb84ef34a50bf06642ee2ba03c5
W20260808 07:05:55.349833 31423 leader_election.cc:341] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eae488a44a4a4e109c01f36910a08257 (127.13.207.2:38893): Not found: Tablet not found: 3724adb84ef34a50bf06642ee2ba03c5
I20260808 07:05:55.349925 31423 leader_election.cc:302] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 10bff4202980420b9b474ee77febc8cd; no voters: c703c80db1e6405699d427084f25a688, eae488a44a4a4e109c01f36910a08257
I20260808 07:05:55.350000 31864 raft_consensus.cc:2726] T 3724adb84ef34a50bf06642ee2ba03c5 P 10bff4202980420b9b474ee77febc8cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:05:55.449873 31865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:55.449571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.3:37531)
0808 07:05:55.449769 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:55.449821 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":95,"threads_started":1}
I20260808 07:05:56.253033 31866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:56.252715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.1:51925)
0808 07:05:56.252930 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:56.252981 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":91,"threads_started":1}
I20260808 07:05:56.309402 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:56.309115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.2:60339)
0808 07:05:56.309294 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:56.309341 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":69,"threads_started":1}
I20260808 07:05:56.450938 31868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:56.450563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.3:55771)
0808 07:05:56.450828 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:56.450887 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":71,"threads_started":1}
I20260808 07:05:57.253993 31869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:57.253706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.1:47843)
0808 07:05:57.253883 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:57.253939 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":75,"threads_started":1}
I20260808 07:05:57.310367 31870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:57.310067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.2:53425)
0808 07:05:57.310234 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:57.310281 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":121,"threads_started":1}
I20260808 07:05:57.451757 31871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:57.451481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.3:41421)
0808 07:05:57.451665 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:57.451707 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":75,"threads_started":1}
I20260808 07:05:58.255234 31872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:58.254856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.1:51583)
0808 07:05:58.255141 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:58.255184 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":169,"threads_started":1}
I20260808 07:05:58.311239 31873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:58.310924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.2:47087)
0808 07:05:58.311140 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:58.311187 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":127,"threads_started":1}
I20260808 07:05:58.452661 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:58.452372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.3:44861)
0808 07:05:58.452551 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:58.452607 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":83,"threads_started":1}
I20260808 07:05:59.256345 31875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:59.256024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.1:56113)
0808 07:05:59.256242 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:59.256294 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":109,"threads_started":1}
I20260808 07:05:59.312271 31876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:59.311971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.2:35651)
0808 07:05:59.312169 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:59.312220 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":71,"threads_started":1}
I20260808 07:05:59.452519 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 31367
I20260808 07:05:59.455022 31877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:05:59.453979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35651 (local address 127.13.207.3:40633)
0808 07:05:59.454911 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0808 07:05:59.454958 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.62:35651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":908,"thread_start_us":65,"threads_started":1}
I20260808 07:05:59.460027 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 31546
I20260808 07:05:59.466516 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 31684
2026-08-08T07:05:59Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58950 ms)
[----------] 30 tests from SecurityITest (266417 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:05:59 dist-test-slave-8tdl krb5kdc[31881](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:05:59 dist-test-slave-8tdl krb5kdc[31881](info): set up 2 sockets
Aug 08 07:05:59 dist-test-slave-8tdl krb5kdc[31881](info): commencing operation
krb5kdc: starting...
W20260808 07:06:00.620545 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:06:00 dist-test-slave-8tdl krb5kdc[31881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:06:00.770953 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:43497
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:36293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:43497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:00.865137 31895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:00.865438 31895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:00.865469 31895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:00.866873 31895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:00.866935 31895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:00.866950 31895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:00.866964 31895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:00.866983 31895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:00.868718 31895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:36293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:43497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:43497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:00.869031 31895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:00.869319 31895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:06:00.876021 31895 server_base.cc:1034] running on GCE node
W20260808 07:06:00.876096 31903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:00.875962 31901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:00.875954 31900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:00.876784 31895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:00.877564 31895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:00.878772 31895 hybrid_clock.cc:648] HybridClock initialized: now 1786172760878757 us; error 35 us; skew 500 ppm
Aug 08 07:06:00 dist-test-slave-8tdl krb5kdc[31881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172760, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:00.881922 31895 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:06:00.882581 31895 webserver.cc:460] Webserver started at http://127.13.207.62:46241/ using document root <none> and password file <none>
I20260808 07:06:00.882880 31895 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:00.882961 31895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:00.883127 31895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:00.884327 31895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "c9b88ed2e2ce4b09b7895c76deef4235"
format_stamp: "Formatted at 2026-08-08 07:06:00 on dist-test-slave-8tdl"
server_key: "d826eb31a10a4970b244e3737c9f6d3d"
server_key_iv: "594e5583677ff41fdf1a916d6132c5fe"
server_key_version: "encryptionkey@0"
I20260808 07:06:00.884756 31895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "c9b88ed2e2ce4b09b7895c76deef4235"
format_stamp: "Formatted at 2026-08-08 07:06:00 on dist-test-slave-8tdl"
server_key: "d826eb31a10a4970b244e3737c9f6d3d"
server_key_iv: "594e5583677ff41fdf1a916d6132c5fe"
server_key_version: "encryptionkey@0"
I20260808 07:06:00.886404 31895 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:06:00.887534 31910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:00.887842 31895 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:00.888020 31895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "c9b88ed2e2ce4b09b7895c76deef4235"
format_stamp: "Formatted at 2026-08-08 07:06:00 on dist-test-slave-8tdl"
server_key: "d826eb31a10a4970b244e3737c9f6d3d"
server_key_iv: "594e5583677ff41fdf1a916d6132c5fe"
server_key_version: "encryptionkey@0"
I20260808 07:06:00.888128 31895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:00.917726 31895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:00.920670 31895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:00.920886 31895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:00.925877 31895 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:43497
I20260808 07:06:00.925920 31964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:43497 every 8 connection(s)
I20260808 07:06:00.926318 31895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:00.927529 31965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:00.928355 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 31895
I20260808 07:06:00.928493 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:00.928694 14140 external_mini_cluster.cc:1428] Setting key f20cc11b8b20635a986ec95956b54717
I20260808 07:06:00.931237 31965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235: Bootstrap starting.
I20260808 07:06:00.932045 31965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:00.932415 31965 log.cc:824] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:00.933219 31965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235: No bootstrap required, opened a new log
I20260808 07:06:00.935878 31965 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9b88ed2e2ce4b09b7895c76deef4235" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 43497 } }
I20260808 07:06:00.936038 31965 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:00.936060 31965 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9b88ed2e2ce4b09b7895c76deef4235, State: Initialized, Role: FOLLOWER
I20260808 07:06:00.936208 31965 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9b88ed2e2ce4b09b7895c76deef4235" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 43497 } }
I20260808 07:06:00.936295 31965 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:00.936354 31965 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:00.936407 31965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 07:06:00 dist-test-slave-8tdl krb5kdc[31881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:06:00.937103 31965 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9b88ed2e2ce4b09b7895c76deef4235" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 43497 } }
I20260808 07:06:00.937209 31965 leader_election.cc:302] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9b88ed2e2ce4b09b7895c76deef4235; no voters:
I20260808 07:06:00.937466 31965 leader_election.cc:288] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:00.937510 31970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:00.937767 31970 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235 [term 1 LEADER]: Becoming Leader. State: Replica: c9b88ed2e2ce4b09b7895c76deef4235, State: Running, Role: LEADER
I20260808 07:06:00.937853 31965 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:00.937994 31970 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9b88ed2e2ce4b09b7895c76deef4235" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 43497 } }
I20260808 07:06:00.938470 31971 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9b88ed2e2ce4b09b7895c76deef4235" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9b88ed2e2ce4b09b7895c76deef4235" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 43497 } } }
I20260808 07:06:00.938556 31971 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:00.938787 31972 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9b88ed2e2ce4b09b7895c76deef4235. Latest consensus state: current_term: 1 leader_uuid: "c9b88ed2e2ce4b09b7895c76deef4235" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9b88ed2e2ce4b09b7895c76deef4235" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 43497 } } }
I20260808 07:06:00.939553 31972 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235 [sys.catalog]: This master's current role is: LEADER
W20260808 07:06:00.939983 31986 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:06:00.940055 31986 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:06:00.940117 31978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:00.940346 31978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:00.942131 31968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:00.929431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53538 (local address 127.13.207.62:43497)
0808 07:06:00.929654 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:00.929665 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:00.929699 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:06:00.930320 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:00.930403 (+ 83us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:00.930477 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:00.930637 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:00.931424 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:00.932815 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:00.933734 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:00.934366 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:00.934492 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:00.937562 (+ 3070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:00.937572 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:00.937573 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:00.937607 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:00.940336 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:00.940747 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:00.940750 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:00.940751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:00.940871 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:00.941106 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:00.941107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:00.941108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:00.941423 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:00.941582 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:00.941753 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 07:06:00.941876 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":115,"threads_started":1}
I20260808 07:06:00.942602 31978 catalog_manager.cc:1347] Generated new cluster ID: f247700e2065473ab262309e28d0b9c9
I20260808 07:06:00.942663 31978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:00.981802 31978 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:00.982730 31978 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:00.998660 31978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235: Generated new TSK 0
I20260808 07:06:00.998957 31978 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
I20260808 07:06:01.039677 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:43497
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:36293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:01.134651 31993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:01.135002 31993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:01.135043 31993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:01.136466 31993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:01.136525 31993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:01.136557 31993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:06:01.138322 31993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:36293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:43497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:01.138729 31993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:01.139014 31993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:01.144662 31999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:01.144728 32001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:01.144728 31998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:01.144917 31993 server_base.cc:1034] running on GCE node
I20260808 07:06:01.145233 31993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:01.145539 31993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:01.146785 31993 hybrid_clock.cc:648] HybridClock initialized: now 1786172761146770 us; error 36 us; skew 500 ppm
Aug 08 07:06:01 dist-test-slave-8tdl krb5kdc[31881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172761, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:01.149641 31993 init.cc:377] Logged in from keytab as kudu/127.13.207.1@KRBTEST.COM (short username kudu)
I20260808 07:06:01.150146 31993 webserver.cc:460] Webserver started at http://127.13.207.1:38851/ using document root <none> and password file <none>
I20260808 07:06:01.150370 31993 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:01.150414 31993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:01.150511 31993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:01.151921 31993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "69718200a4174c86863e2a80cc11fb5a"
format_stamp: "Formatted at 2026-08-08 07:06:01 on dist-test-slave-8tdl"
server_key: "5a6d70880f3c3d4b629f9cb386327ebd"
server_key_iv: "583633acd7d34708367f88ec8fc5a95e"
server_key_version: "encryptionkey@0"
I20260808 07:06:01.152287 31993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "69718200a4174c86863e2a80cc11fb5a"
format_stamp: "Formatted at 2026-08-08 07:06:01 on dist-test-slave-8tdl"
server_key: "5a6d70880f3c3d4b629f9cb386327ebd"
server_key_iv: "583633acd7d34708367f88ec8fc5a95e"
server_key_version: "encryptionkey@0"
I20260808 07:06:01.153842 31993 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:06:01.154901 32009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:01.155205 31993 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:01.155323 31993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "69718200a4174c86863e2a80cc11fb5a"
format_stamp: "Formatted at 2026-08-08 07:06:01 on dist-test-slave-8tdl"
server_key: "5a6d70880f3c3d4b629f9cb386327ebd"
server_key_iv: "583633acd7d34708367f88ec8fc5a95e"
server_key_version: "encryptionkey@0"
I20260808 07:06:01.155416 31993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:01.174484 31993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:01.176998 31993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:01.177184 31993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:01.177467 31993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:01.177845 31993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:01.177883 31993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:01.177908 31993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:01.177922 31993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:01.184629 31993 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:32825
I20260808 07:06:01.184736 32131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:32825 every 8 connection(s)
I20260808 07:06:01.185088 31993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:06:01.186570 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 31993
I20260808 07:06:01.186661 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:06:01.186913 14140 external_mini_cluster.cc:1428] Setting key 70475aa22516176148b5b699ac185497
Aug 08 07:06:01 dist-test-slave-8tdl krb5kdc[31881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172761, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.207.2@KRBTEST.COM; defaulting to no policy
I20260808 07:06:01.197310 31968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:01.186030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:36487 (local address 127.13.207.62:43497)
0808 07:06:01.186093 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:01.186095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:01.186316 (+ 221us) server_negotiation.cc:407] Connection header received
0808 07:06:01.187144 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:01.187145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:01.187158 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:01.187215 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:01.188236 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.188767 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.190274 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.190545 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.190743 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:01.193437 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:01.193439 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:01.193440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:01.193450 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:01.195478 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:01.196015 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:01.196017 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:01.196018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:01.196091 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:01.196413 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:01.196414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:01.196414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:01.196580 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:01.196668 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:01.197033 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 07:06:01.197165 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:06:01.197556 32134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:01.185989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:43497 (local address 127.13.207.1:36487)
0808 07:06:01.186179 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:01.186223 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:01.187022 (+ 799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:01.187298 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:01.187302 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:01.187920 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:01.188126 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:01.188131 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.188874 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.188875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:01.190152 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:01.190155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.190735 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.190736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:01.190896 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:01.191533 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:01.191544 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:01.193323 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:01.195626 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:01.195636 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:01.195639 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:01.195918 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:01.196208 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:01.196209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:01.196209 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:01.196337 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:01.196694 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:01.196702 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:01.196877 (+ 175us) client_negotiation.cc:769] Sending connection context
0808 07:06:01.197052 (+ 175us) client_negotiation.cc:240] Negotiation successful
0808 07:06:01.197292 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":93,"threads_started":1}
I20260808 07:06:01.198140 32132 heartbeater.cc:347] Connected to a master server at 127.13.207.62:43497
I20260808 07:06:01.198256 32132 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:01.198513 32132 heartbeater.cc:511] Master 127.13.207.62:43497 requested a full tablet report, sending...
I20260808 07:06:01.199213 31928 ts_manager.cc:194] Registered new tserver with Master: 69718200a4174c86863e2a80cc11fb5a (127.13.207.1:32825)
I20260808 07:06:01.199909 31928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:36487
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
I20260808 07:06:01.222997 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:43497
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:36293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:01.309721 32137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:01.309988 32137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:01.310040 32137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:01.311302 32137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:01.311384 32137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:01.311431 32137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:06:01.313060 32137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:36293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:43497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:01.313376 32137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:01.313655 32137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:01.319691 32143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:01.319626 32146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:01.319832 32142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:01.320194 32137 server_base.cc:1034] running on GCE node
I20260808 07:06:01.320436 32137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:01.320727 32137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:01.321924 32137 hybrid_clock.cc:648] HybridClock initialized: now 1786172761321937 us; error 61 us; skew 500 ppm
Aug 08 07:06:01 dist-test-slave-8tdl krb5kdc[31881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172761, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:01.324972 32137 init.cc:377] Logged in from keytab as kudu/127.13.207.2@KRBTEST.COM (short username kudu)
I20260808 07:06:01.325492 32137 webserver.cc:460] Webserver started at http://127.13.207.2:44885/ using document root <none> and password file <none>
I20260808 07:06:01.325788 32137 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:01.325857 32137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:01.326005 32137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:01.327178 32137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "28fef4759f9342adb81f687d319093c2"
format_stamp: "Formatted at 2026-08-08 07:06:01 on dist-test-slave-8tdl"
server_key: "cc9b652eb7ed606d932b142c08adb90c"
server_key_iv: "83a641431b30aed1d731d4687b193c74"
server_key_version: "encryptionkey@0"
I20260808 07:06:01.327545 32137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "28fef4759f9342adb81f687d319093c2"
format_stamp: "Formatted at 2026-08-08 07:06:01 on dist-test-slave-8tdl"
server_key: "cc9b652eb7ed606d932b142c08adb90c"
server_key_iv: "83a641431b30aed1d731d4687b193c74"
server_key_version: "encryptionkey@0"
I20260808 07:06:01.328946 32137 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:06:01.329756 32153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:01.329991 32137 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:01.330098 32137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "28fef4759f9342adb81f687d319093c2"
format_stamp: "Formatted at 2026-08-08 07:06:01 on dist-test-slave-8tdl"
server_key: "cc9b652eb7ed606d932b142c08adb90c"
server_key_iv: "83a641431b30aed1d731d4687b193c74"
server_key_version: "encryptionkey@0"
I20260808 07:06:01.330174 32137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:01.351504 32137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:01.354350 32137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:01.354612 32137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:01.354938 32137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:01.355316 32137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:01.355372 32137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:01.355415 32137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:01.355445 32137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:01.362287 32137 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:41661
I20260808 07:06:01.362433 32276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:41661 every 8 connection(s)
I20260808 07:06:01.362807 32137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:06:01 dist-test-slave-8tdl krb5kdc[31881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172761, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:06:01.370201 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 32137
I20260808 07:06:01.370342 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:06:01.370601 14140 external_mini_cluster.cc:1428] Setting key e6b14f049dc74a47b9013e0622879326
I20260808 07:06:01.373762 31968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:01.363678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:47125 (local address 127.13.207.62:43497)
0808 07:06:01.363759 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:01.363761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:01.364005 (+ 244us) server_negotiation.cc:407] Connection header received
0808 07:06:01.364536 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:01.364537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:01.364549 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:01.364609 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:01.365710 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.366244 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.367405 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.367630 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.367702 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:01.370304 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:01.370306 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:01.370306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:01.370316 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:01.372202 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:01.372687 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:01.372688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:01.372688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:01.372755 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:01.373051 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:01.373052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:01.373052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:01.373196 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:01.373271 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:01.373570 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 07:06:01.373663 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:06:01.374006 32280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:01.363625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:43497 (local address 127.13.207.2:47125)
0808 07:06:01.363822 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:01.363862 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:01.364431 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:01.364666 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:01.364669 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:01.365354 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:01.365591 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:01.365598 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.366385 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.366386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:01.367292 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:01.367295 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.367753 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.367754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:01.367877 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:01.368507 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:01.368516 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:01.370191 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:01.372303 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:01.372316 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:01.372319 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:01.372603 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:01.372829 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:01.372830 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:01.372830 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:01.372971 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:01.373262 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:01.373270 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:01.373421 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 07:06:01.373530 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 07:06:01.373721 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":115,"threads_started":1}
I20260808 07:06:01.374472 32278 heartbeater.cc:347] Connected to a master server at 127.13.207.62:43497
I20260808 07:06:01.374608 32278 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:01.374890 32278 heartbeater.cc:511] Master 127.13.207.62:43497 requested a full tablet report, sending...
I20260808 07:06:01.375311 31928 ts_manager.cc:194] Registered new tserver with Master: 28fef4759f9342adb81f687d319093c2 (127.13.207.2:41661)
I20260808 07:06:01.375720 31928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:47125
WARNING: no policy specified for kudu/127.13.207.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
I20260808 07:06:01.407604 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:43497
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:36293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:01.496057 32283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:01.496291 32283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:01.496318 32283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:01.497538 32283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:01.497592 32283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:01.497622 32283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:06:01.499432 32283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:36293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:43497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:01.499718 32283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:01.499966 32283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:01.505735 32290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:01.505736 32292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:01.505736 32289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:01.505779 32283 server_base.cc:1034] running on GCE node
I20260808 07:06:01.506264 32283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:01.506578 32283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:01.507747 32283 hybrid_clock.cc:648] HybridClock initialized: now 1786172761507734 us; error 41 us; skew 500 ppm
Aug 08 07:06:01 dist-test-slave-8tdl krb5kdc[31881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172761, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:01.510766 32283 init.cc:377] Logged in from keytab as kudu/127.13.207.3@KRBTEST.COM (short username kudu)
I20260808 07:06:01.511472 32283 webserver.cc:460] Webserver started at http://127.13.207.3:36829/ using document root <none> and password file <none>
I20260808 07:06:01.511755 32283 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:01.511828 32283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:01.512012 32283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:01.513201 32283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "ebdedaa9276e469ab2e5099fd6b7351a"
format_stamp: "Formatted at 2026-08-08 07:06:01 on dist-test-slave-8tdl"
server_key: "06b2229c798a7fa731c52aff21dd222b"
server_key_iv: "ce59745488af745d30b0bcf4c004abf6"
server_key_version: "encryptionkey@0"
I20260808 07:06:01.513674 32283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "ebdedaa9276e469ab2e5099fd6b7351a"
format_stamp: "Formatted at 2026-08-08 07:06:01 on dist-test-slave-8tdl"
server_key: "06b2229c798a7fa731c52aff21dd222b"
server_key_iv: "ce59745488af745d30b0bcf4c004abf6"
server_key_version: "encryptionkey@0"
I20260808 07:06:01.515493 32283 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 07:06:01.516605 32299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:01.516970 32283 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:01.517093 32283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "ebdedaa9276e469ab2e5099fd6b7351a"
format_stamp: "Formatted at 2026-08-08 07:06:01 on dist-test-slave-8tdl"
server_key: "06b2229c798a7fa731c52aff21dd222b"
server_key_iv: "ce59745488af745d30b0bcf4c004abf6"
server_key_version: "encryptionkey@0"
I20260808 07:06:01.517180 32283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:01.540351 32283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:01.543469 32283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:01.543737 32283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:01.544180 32283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:01.544625 32283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:01.544677 32283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:01.544703 32283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:01.544718 32283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:01.552879 32283 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:43671
I20260808 07:06:01.553066 32417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:43671 every 8 connection(s)
I20260808 07:06:01.553359 32283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:06:01.554549 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 32283
I20260808 07:06:01.554670 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:06:01.554915 14140 external_mini_cluster.cc:1428] Setting key 2c9808b653a0558d1bef00d50bf70801
Aug 08 07:06:01 dist-test-slave-8tdl krb5kdc[31881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172761, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:06:01.565919 31968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:01.554283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:54027 (local address 127.13.207.62:43497)
0808 07:06:01.554404 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:01.554406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:01.554643 (+ 237us) server_negotiation.cc:407] Connection header received
0808 07:06:01.555333 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:01.555335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:01.555348 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:01.555397 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:01.556468 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.556996 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.559193 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.559425 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.559517 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:01.562544 (+ 3027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:01.562546 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:01.562546 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:01.562555 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:01.564355 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:01.564873 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:01.564874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:01.564875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:01.564941 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:01.565211 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:01.565212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:01.565212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:01.565344 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:01.565423 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:01.565706 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 07:06:01.565800 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 07:06:01.566229 32420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:01.554245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:43497 (local address 127.13.207.3:54027)
0808 07:06:01.554468 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:01.554520 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:01.555218 (+ 698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:01.555483 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:01.555488 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:01.556117 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:01.556333 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:01.556338 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.557119 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.557120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:01.559038 (+ 1918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:01.559044 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.559531 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.559532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:01.559668 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:01.560323 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:01.560333 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:01.562323 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:01.564494 (+ 2171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:01.564506 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:01.564509 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:01.564781 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:01.565012 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:01.565013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:01.565014 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:01.565140 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:01.565412 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:01.565420 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:01.565571 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 07:06:01.565685 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 07:06:01.565873 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":123,"threads_started":1}
I20260808 07:06:01.566829 32418 heartbeater.cc:347] Connected to a master server at 127.13.207.62:43497
I20260808 07:06:01.566946 32418 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:01.567149 32418 heartbeater.cc:511] Master 127.13.207.62:43497 requested a full tablet report, sending...
I20260808 07:06:01.567574 31928 ts_manager.cc:194] Registered new tserver with Master: ebdedaa9276e469ab2e5099fd6b7351a (127.13.207.3:43671)
I20260808 07:06:01.567965 31928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:54027
I20260808 07:06:01.578066 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:06:01.586730 31968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:01.579663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53552 (local address 127.13.207.62:43497)
0808 07:06:01.579753 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:01.579756 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:01.580087 (+ 331us) server_negotiation.cc:407] Connection header received
0808 07:06:01.580126 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:01.580127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:01.580138 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:01.580196 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:01.580797 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.581319 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.582054 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.582288 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.582428 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:01.583719 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:01.583720 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:01.583720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:01.583727 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:01.585203 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:01.585620 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:01.585622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:01.585622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:01.585692 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:01.585879 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:01.585880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:01.585880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:01.586108 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:01.586199 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:01.586418 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 07:06:01.586509 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 07:06:01.588862 31928 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:06:01.590807 31928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:01.598246 32433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:01.593973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:41661 (local address 127.0.0.1:56674)
0808 07:06:01.594235 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:01.594248 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:01.594307 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:01.594597 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:01.594597 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:01.594599 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:01.594721 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:01.594724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.596188 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.596188 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:01.596929 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:01.596931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.597808 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.597809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:01.597899 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:01.597914 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 07:06:01.597983 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 07:06:01.598045 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":7,"thread_start_us":68,"threads_started":1}
I20260808 07:06:01.598388 32432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:01.593901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49086 (local address 127.13.207.1:32825)
0808 07:06:01.594462 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:01.594467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:01.594478 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:06:01.594520 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:01.594523 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:01.594540 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:01.594618 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:01.594916 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.595652 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.597129 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.597908 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.597992 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:01.598055 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:01.598241 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 07:06:01.598292 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":121,"threads_started":1}
I20260808 07:06:01.598351 32430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:01.593758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:32825 (local address 127.0.0.1:49086)
0808 07:06:01.594143 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:01.594152 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:01.594195 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:01.594743 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:01.594744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:01.594746 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:01.594811 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:01.594813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.595765 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.595766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:01.597024 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:01.597027 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.598000 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.598001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:01.598064 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:01.598068 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:01.598218 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 07:06:01.598280 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":99,"threads_started":1}
I20260808 07:06:01.598476 32431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:01.593755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:43671 (local address 127.0.0.1:37256)
0808 07:06:01.593918 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:01.593971 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:01.594048 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:01.594580 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:01.594583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:01.594586 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:01.594721 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:01.594724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.596150 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.596151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:01.597216 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:01.597218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.598239 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.598240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:01.598310 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:01.598314 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:01.598347 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 07:06:01.598393 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":8,"thread_start_us":68,"threads_started":1}
I20260808 07:06:01.598839 32434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:01.593946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37256 (local address 127.13.207.3:43671)
0808 07:06:01.594297 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:01.594302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:01.594327 (+ 25us) server_negotiation.cc:407] Connection header received
0808 07:06:01.594378 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:01.594380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:01.594398 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:01.594810 (+ 412us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:01.594895 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.596032 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.597313 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.598062 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.598571 (+ 509us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:01.598625 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:01.598647 (+ 22us) server_negotiation.cc:299] Negotiation successful
0808 07:06:01.598738 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":107,"threads_started":1}
I20260808 07:06:01.599512 32435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:01.594198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56674 (local address 127.13.207.2:41661)
0808 07:06:01.594416 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:01.594422 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:01.594432 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:06:01.594475 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:01.594477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:01.594491 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:01.594809 (+ 318us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:01.594902 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.596044 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.597026 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.597712 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.597806 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:01.597870 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:01.599300 (+ 1430us) server_negotiation.cc:299] Negotiation successful
0808 07:06:01.599358 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":111,"threads_started":1}
I20260808 07:06:01.599658 32349 tablet_service.cc:1467] Processing CreateTablet for tablet b5a7b44d1c144fa68e4b0d7050f5a135 (DEFAULT_TABLE table=test-table [id=6f6f95c2f1f24683b1238706c3d9e83d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:01.599658 32061 tablet_service.cc:1467] Processing CreateTablet for tablet b5a7b44d1c144fa68e4b0d7050f5a135 (DEFAULT_TABLE table=test-table [id=6f6f95c2f1f24683b1238706c3d9e83d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:01.600039 32061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5a7b44d1c144fa68e4b0d7050f5a135. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:01.600040 32349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5a7b44d1c144fa68e4b0d7050f5a135. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:01.600255 32204 tablet_service.cc:1467] Processing CreateTablet for tablet b5a7b44d1c144fa68e4b0d7050f5a135 (DEFAULT_TABLE table=test-table [id=6f6f95c2f1f24683b1238706c3d9e83d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:01.600593 32204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5a7b44d1c144fa68e4b0d7050f5a135. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:01.602754 32436 tablet_bootstrap.cc:492] T b5a7b44d1c144fa68e4b0d7050f5a135 P 69718200a4174c86863e2a80cc11fb5a: Bootstrap starting.
I20260808 07:06:01.603430 32436 tablet_bootstrap.cc:654] T b5a7b44d1c144fa68e4b0d7050f5a135 P 69718200a4174c86863e2a80cc11fb5a: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:01.603727 32437 tablet_bootstrap.cc:492] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a: Bootstrap starting.
I20260808 07:06:01.604075 32436 log.cc:824] T b5a7b44d1c144fa68e4b0d7050f5a135 P 69718200a4174c86863e2a80cc11fb5a: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:01.604475 32437 tablet_bootstrap.cc:654] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:01.605054 32436 tablet_bootstrap.cc:492] T b5a7b44d1c144fa68e4b0d7050f5a135 P 69718200a4174c86863e2a80cc11fb5a: No bootstrap required, opened a new log
I20260808 07:06:01.605130 32437 log.cc:824] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:01.605155 32436 ts_tablet_manager.cc:1397] T b5a7b44d1c144fa68e4b0d7050f5a135 P 69718200a4174c86863e2a80cc11fb5a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:06:01.606925 32439 tablet_bootstrap.cc:492] T b5a7b44d1c144fa68e4b0d7050f5a135 P 28fef4759f9342adb81f687d319093c2: Bootstrap starting.
I20260808 07:06:01.607100 32436 raft_consensus.cc:348] T b5a7b44d1c144fa68e4b0d7050f5a135 P 69718200a4174c86863e2a80cc11fb5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69718200a4174c86863e2a80cc11fb5a" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32825 } } peers { permanent_uuid: "28fef4759f9342adb81f687d319093c2" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41661 } } peers { permanent_uuid: "ebdedaa9276e469ab2e5099fd6b7351a" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43671 } }
I20260808 07:06:01.607332 32436 raft_consensus.cc:374] T b5a7b44d1c144fa68e4b0d7050f5a135 P 69718200a4174c86863e2a80cc11fb5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:01.607406 32436 raft_consensus.cc:729] T b5a7b44d1c144fa68e4b0d7050f5a135 P 69718200a4174c86863e2a80cc11fb5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69718200a4174c86863e2a80cc11fb5a, State: Initialized, Role: FOLLOWER
I20260808 07:06:01.607544 32436 consensus_queue.cc:260] T b5a7b44d1c144fa68e4b0d7050f5a135 P 69718200a4174c86863e2a80cc11fb5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69718200a4174c86863e2a80cc11fb5a" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32825 } } peers { permanent_uuid: "28fef4759f9342adb81f687d319093c2" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41661 } } peers { permanent_uuid: "ebdedaa9276e469ab2e5099fd6b7351a" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43671 } }
I20260808 07:06:01.607816 32439 tablet_bootstrap.cc:654] T b5a7b44d1c144fa68e4b0d7050f5a135 P 28fef4759f9342adb81f687d319093c2: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:01.607889 32436 ts_tablet_manager.cc:1428] T b5a7b44d1c144fa68e4b0d7050f5a135 P 69718200a4174c86863e2a80cc11fb5a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:06:01.607918 32132 heartbeater.cc:503] Master 127.13.207.62:43497 was elected leader, sending a full tablet report...
I20260808 07:06:01.608456 32439 log.cc:824] T b5a7b44d1c144fa68e4b0d7050f5a135 P 28fef4759f9342adb81f687d319093c2: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:01.609373 32439 tablet_bootstrap.cc:492] T b5a7b44d1c144fa68e4b0d7050f5a135 P 28fef4759f9342adb81f687d319093c2: No bootstrap required, opened a new log
I20260808 07:06:01.609459 32439 ts_tablet_manager.cc:1397] T b5a7b44d1c144fa68e4b0d7050f5a135 P 28fef4759f9342adb81f687d319093c2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:06:01.609503 32437 tablet_bootstrap.cc:492] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a: No bootstrap required, opened a new log
I20260808 07:06:01.609582 32437 ts_tablet_manager.cc:1397] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260808 07:06:01.611585 32439 raft_consensus.cc:348] T b5a7b44d1c144fa68e4b0d7050f5a135 P 28fef4759f9342adb81f687d319093c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69718200a4174c86863e2a80cc11fb5a" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32825 } } peers { permanent_uuid: "28fef4759f9342adb81f687d319093c2" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41661 } } peers { permanent_uuid: "ebdedaa9276e469ab2e5099fd6b7351a" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43671 } }
I20260808 07:06:01.611774 32439 raft_consensus.cc:374] T b5a7b44d1c144fa68e4b0d7050f5a135 P 28fef4759f9342adb81f687d319093c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:01.611681 32437 raft_consensus.cc:348] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69718200a4174c86863e2a80cc11fb5a" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32825 } } peers { permanent_uuid: "28fef4759f9342adb81f687d319093c2" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41661 } } peers { permanent_uuid: "ebdedaa9276e469ab2e5099fd6b7351a" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43671 } }
I20260808 07:06:01.611826 32439 raft_consensus.cc:729] T b5a7b44d1c144fa68e4b0d7050f5a135 P 28fef4759f9342adb81f687d319093c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28fef4759f9342adb81f687d319093c2, State: Initialized, Role: FOLLOWER
I20260808 07:06:01.611873 32437 raft_consensus.cc:374] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:01.611922 32437 raft_consensus.cc:729] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebdedaa9276e469ab2e5099fd6b7351a, State: Initialized, Role: FOLLOWER
I20260808 07:06:01.611971 32439 consensus_queue.cc:260] T b5a7b44d1c144fa68e4b0d7050f5a135 P 28fef4759f9342adb81f687d319093c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69718200a4174c86863e2a80cc11fb5a" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32825 } } peers { permanent_uuid: "28fef4759f9342adb81f687d319093c2" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41661 } } peers { permanent_uuid: "ebdedaa9276e469ab2e5099fd6b7351a" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43671 } }
I20260808 07:06:01.612046 32437 consensus_queue.cc:260] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69718200a4174c86863e2a80cc11fb5a" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32825 } } peers { permanent_uuid: "28fef4759f9342adb81f687d319093c2" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41661 } } peers { permanent_uuid: "ebdedaa9276e469ab2e5099fd6b7351a" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43671 } }
I20260808 07:06:01.612314 32278 heartbeater.cc:503] Master 127.13.207.62:43497 was elected leader, sending a full tablet report...
I20260808 07:06:01.612377 32437 ts_tablet_manager.cc:1428] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:06:01.612407 32418 heartbeater.cc:503] Master 127.13.207.62:43497 was elected leader, sending a full tablet report...
I20260808 07:06:01.612330 32439 ts_tablet_manager.cc:1428] T b5a7b44d1c144fa68e4b0d7050f5a135 P 28fef4759f9342adb81f687d319093c2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260808 07:06:01.613420 32279 tablet.cc:2369] T b5a7b44d1c144fa68e4b0d7050f5a135 P 28fef4759f9342adb81f687d319093c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:06:01.686079 32133 tablet.cc:2369] T b5a7b44d1c144fa68e4b0d7050f5a135 P 69718200a4174c86863e2a80cc11fb5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:06:01.804176 32419 tablet.cc:2369] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:06:01.907328 32444 raft_consensus.cc:482] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:01.907477 32444 raft_consensus.cc:504] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69718200a4174c86863e2a80cc11fb5a" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32825 } } peers { permanent_uuid: "28fef4759f9342adb81f687d319093c2" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41661 } } peers { permanent_uuid: "ebdedaa9276e469ab2e5099fd6b7351a" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43671 } }
I20260808 07:06:01.908057 32444 leader_election.cc:288] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69718200a4174c86863e2a80cc11fb5a (127.13.207.1:32825), 28fef4759f9342adb81f687d319093c2 (127.13.207.2:41661)
I20260808 07:06:01.912276 32445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:01.908470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:41661 (local address 127.13.207.3:35979)
0808 07:06:01.908666 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:01.908674 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:01.908739 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:01.908869 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:01.908870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:01.908872 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:01.909058 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:01.909062 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.909907 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.909908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:01.911018 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:01.911019 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.911955 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.911956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:01.912045 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:01.912051 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:06:01.912108 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 07:06:01.912164 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":133,"threads_started":1}
I20260808 07:06:01.912289 32435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:01.908533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:35979 (local address 127.13.207.2:41661)
0808 07:06:01.908606 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:01.908608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:01.908751 (+ 143us) server_negotiation.cc:407] Connection header received
0808 07:06:01.908785 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:01.908786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:01.908796 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:01.908844 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:01.909164 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.909815 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.911148 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.911791 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.911900 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:01.911913 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:01.912146 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 07:06:01.912203 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:06:01.912336 32432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:01.908408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:38895 (local address 127.13.207.1:32825)
0808 07:06:01.908461 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:01.908463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:01.908475 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:06:01.908621 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:01.908623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:01.908636 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:01.908702 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:01.908929 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.909607 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.911152 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.911853 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.911933 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:01.911943 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:01.912197 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 07:06:01.912253 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 07:06:01.912384 32420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:01.908284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:32825 (local address 127.13.207.3:38895)
0808 07:06:01.908389 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:01.908405 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:01.908481 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:01.908728 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:01.908729 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:01.908731 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:01.908822 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:01.908825 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.909695 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.909696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:01.910963 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:01.910966 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.911987 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.911988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:01.912080 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:01.912083 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:01.912171 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 07:06:01.912221 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 07:06:01.913136 32226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5a7b44d1c144fa68e4b0d7050f5a135" candidate_uuid: "ebdedaa9276e469ab2e5099fd6b7351a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28fef4759f9342adb81f687d319093c2" is_pre_election: true
I20260808 07:06:01.913136 32083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5a7b44d1c144fa68e4b0d7050f5a135" candidate_uuid: "ebdedaa9276e469ab2e5099fd6b7351a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69718200a4174c86863e2a80cc11fb5a" is_pre_election: true
I20260808 07:06:01.913342 32226 raft_consensus.cc:2454] T b5a7b44d1c144fa68e4b0d7050f5a135 P 28fef4759f9342adb81f687d319093c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebdedaa9276e469ab2e5099fd6b7351a in term 0.
I20260808 07:06:01.913341 32083 raft_consensus.cc:2454] T b5a7b44d1c144fa68e4b0d7050f5a135 P 69718200a4174c86863e2a80cc11fb5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebdedaa9276e469ab2e5099fd6b7351a in term 0.
I20260808 07:06:01.913657 32301 leader_election.cc:302] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69718200a4174c86863e2a80cc11fb5a, ebdedaa9276e469ab2e5099fd6b7351a; no voters:
I20260808 07:06:01.913844 32444 raft_consensus.cc:2781] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:01.913942 32444 raft_consensus.cc:482] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:01.913975 32444 raft_consensus.cc:3037] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:01.915028 32444 raft_consensus.cc:504] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69718200a4174c86863e2a80cc11fb5a" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32825 } } peers { permanent_uuid: "28fef4759f9342adb81f687d319093c2" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41661 } } peers { permanent_uuid: "ebdedaa9276e469ab2e5099fd6b7351a" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43671 } }
I20260808 07:06:01.915215 32444 leader_election.cc:288] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a [CANDIDATE]: Term 1 election: Requested vote from peers 69718200a4174c86863e2a80cc11fb5a (127.13.207.1:32825), 28fef4759f9342adb81f687d319093c2 (127.13.207.2:41661)
I20260808 07:06:01.915583 32083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5a7b44d1c144fa68e4b0d7050f5a135" candidate_uuid: "ebdedaa9276e469ab2e5099fd6b7351a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69718200a4174c86863e2a80cc11fb5a"
I20260808 07:06:01.915597 32226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5a7b44d1c144fa68e4b0d7050f5a135" candidate_uuid: "ebdedaa9276e469ab2e5099fd6b7351a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28fef4759f9342adb81f687d319093c2"
I20260808 07:06:01.915696 32226 raft_consensus.cc:3037] T b5a7b44d1c144fa68e4b0d7050f5a135 P 28fef4759f9342adb81f687d319093c2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:01.915696 32083 raft_consensus.cc:3037] T b5a7b44d1c144fa68e4b0d7050f5a135 P 69718200a4174c86863e2a80cc11fb5a [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:01.916746 32226 raft_consensus.cc:2454] T b5a7b44d1c144fa68e4b0d7050f5a135 P 28fef4759f9342adb81f687d319093c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebdedaa9276e469ab2e5099fd6b7351a in term 1.
I20260808 07:06:01.916746 32083 raft_consensus.cc:2454] T b5a7b44d1c144fa68e4b0d7050f5a135 P 69718200a4174c86863e2a80cc11fb5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebdedaa9276e469ab2e5099fd6b7351a in term 1.
I20260808 07:06:01.916988 32301 leader_election.cc:302] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28fef4759f9342adb81f687d319093c2, ebdedaa9276e469ab2e5099fd6b7351a; no voters:
I20260808 07:06:01.917100 32444 raft_consensus.cc:2781] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:01.917280 32444 raft_consensus.cc:686] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a [term 1 LEADER]: Becoming Leader. State: Replica: ebdedaa9276e469ab2e5099fd6b7351a, State: Running, Role: LEADER
I20260808 07:06:01.917409 32444 consensus_queue.cc:237] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69718200a4174c86863e2a80cc11fb5a" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32825 } } peers { permanent_uuid: "28fef4759f9342adb81f687d319093c2" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41661 } } peers { permanent_uuid: "ebdedaa9276e469ab2e5099fd6b7351a" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43671 } }
I20260808 07:06:01.918390 31928 catalog_manager.cc:5495] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a reported cstate change: term changed from 0 to 1, leader changed from <none> to ebdedaa9276e469ab2e5099fd6b7351a (127.13.207.3). New cstate: current_term: 1 leader_uuid: "ebdedaa9276e469ab2e5099fd6b7351a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69718200a4174c86863e2a80cc11fb5a" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28fef4759f9342adb81f687d319093c2" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebdedaa9276e469ab2e5099fd6b7351a" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43671 } health_report { overall_health: HEALTHY } } }
I20260808 07:06:01.959154 32434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:01.956538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37272 (local address 127.13.207.3:43671)
0808 07:06:01.956615 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:01.956617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:01.956624 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:06:01.956664 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:01.956665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:01.956685 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:01.956760 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:01.956967 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.957469 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.958257 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:01.958461 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:01.958538 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:01.958755 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:01.958883 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:01.958982 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:01.959007 (+ 25us) server_negotiation.cc:299] Negotiation successful
0808 07:06:01.959033 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:06:01.961180 32226 raft_consensus.cc:1261] T b5a7b44d1c144fa68e4b0d7050f5a135 P 28fef4759f9342adb81f687d319093c2 [term 1 FOLLOWER]: Refusing update from remote peer ebdedaa9276e469ab2e5099fd6b7351a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:01.961206 32083 raft_consensus.cc:1261] T b5a7b44d1c144fa68e4b0d7050f5a135 P 69718200a4174c86863e2a80cc11fb5a [term 1 FOLLOWER]: Refusing update from remote peer ebdedaa9276e469ab2e5099fd6b7351a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:01.961613 32446 consensus_queue.cc:1035] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a [LEADER]: Connected to new peer: Peer: permanent_uuid: "69718200a4174c86863e2a80cc11fb5a" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 32825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:01.961843 32444 consensus_queue.cc:1035] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a [LEADER]: Connected to new peer: Peer: permanent_uuid: "28fef4759f9342adb81f687d319093c2" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:01.964389 32449 mvcc.cc:205] Tried to move back new op lower bound from 7316163632989786112 to 7316163632814440448. Current Snapshot: MvccSnapshot[applied={T|T < 7316163632989786112}]
I20260808 07:06:01.964803 32450 mvcc.cc:205] Tried to move back new op lower bound from 7316163632989786112 to 7316163632814440448. Current Snapshot: MvccSnapshot[applied={T|T < 7316163632989786112 or (T in {7316163632989786112})}]
I20260808 07:06:01.966236 32452 mvcc.cc:205] Tried to move back new op lower bound from 7316163632989786112 to 7316163632814440448. Current Snapshot: MvccSnapshot[applied={T|T < 7316163632989786112}]
I20260808 07:06:01.966965 31928 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:01.967085 31928 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:01.967912 31928 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c9b88ed2e2ce4b09b7895c76deef4235: Sending DeleteTablet for 3 replicas of tablet b5a7b44d1c144fa68e4b0d7050f5a135
I20260808 07:06:01.968331 32349 tablet_service.cc:1514] Processing DeleteTablet for tablet b5a7b44d1c144fa68e4b0d7050f5a135 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:01 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:37256
I20260808 07:06:01.968381 32061 tablet_service.cc:1514] Processing DeleteTablet for tablet b5a7b44d1c144fa68e4b0d7050f5a135 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:01 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:49086
I20260808 07:06:01.968590 32204 tablet_service.cc:1514] Processing DeleteTablet for tablet b5a7b44d1c144fa68e4b0d7050f5a135 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:01 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:56674
I20260808 07:06:01.968724 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 31993
I20260808 07:06:01.968883 32462 tablet_replica.cc:330] T b5a7b44d1c144fa68e4b0d7050f5a135 P 28fef4759f9342adb81f687d319093c2: stopping tablet replica
I20260808 07:06:01.969012 32462 raft_consensus.cc:2229] T b5a7b44d1c144fa68e4b0d7050f5a135 P 28fef4759f9342adb81f687d319093c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:01.969276 32462 raft_consensus.cc:2258] T b5a7b44d1c144fa68e4b0d7050f5a135 P 28fef4759f9342adb81f687d319093c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:01.969555 32460 tablet_replica.cc:330] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a: stopping tablet replica
I20260808 07:06:01.969720 32460 raft_consensus.cc:2229] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:06:01.969875 32460 raft_consensus.cc:2258] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:01.970429 32460 ts_tablet_manager.cc:1905] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:01.971101 32462 ts_tablet_manager.cc:1905] T b5a7b44d1c144fa68e4b0d7050f5a135 P 28fef4759f9342adb81f687d319093c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:01.973269 32462 ts_tablet_manager.cc:1918] T b5a7b44d1c144fa68e4b0d7050f5a135 P 28fef4759f9342adb81f687d319093c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:01.973367 32462 log.cc:1196] T b5a7b44d1c144fa68e4b0d7050f5a135 P 28fef4759f9342adb81f687d319093c2: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-1/wal/wals/b5a7b44d1c144fa68e4b0d7050f5a135
I20260808 07:06:01.973726 32462 ts_tablet_manager.cc:1939] T b5a7b44d1c144fa68e4b0d7050f5a135 P 28fef4759f9342adb81f687d319093c2: Deleting consensus metadata
I20260808 07:06:01.974220 31912 catalog_manager.cc:4841] TS 28fef4759f9342adb81f687d319093c2 (127.13.207.2:41661): tablet b5a7b44d1c144fa68e4b0d7050f5a135 (table test-table [id=6f6f95c2f1f24683b1238706c3d9e83d]) successfully deleted
I20260808 07:06:01.974514 32460 ts_tablet_manager.cc:1918] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:01.974607 32460 log.cc:1196] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/b5a7b44d1c144fa68e4b0d7050f5a135
I20260808 07:06:01.975003 32460 ts_tablet_manager.cc:1939] T b5a7b44d1c144fa68e4b0d7050f5a135 P ebdedaa9276e469ab2e5099fd6b7351a: Deleting consensus metadata
I20260808 07:06:01.975488 31914 catalog_manager.cc:4841] TS ebdedaa9276e469ab2e5099fd6b7351a (127.13.207.3:43671): tablet b5a7b44d1c144fa68e4b0d7050f5a135 (table test-table [id=6f6f95c2f1f24683b1238706c3d9e83d]) successfully deleted
W20260808 07:06:01.978425 31912 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.207.1:32825 (error 108)
I20260808 07:06:01.978811 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 32137
I20260808 07:06:01.979017 32430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:01.978863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:32825 (local address 127.0.0.1:49100)
0808 07:06:01.978928 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:01.978968 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.1:32825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 07:06:01.981815 31912 catalog_manager.cc:4584] TS 69718200a4174c86863e2a80cc11fb5a (127.13.207.1:32825): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b5a7b44d1c144fa68e4b0d7050f5a135: Network error: Client connection negotiation failed: client connection to 127.13.207.1:32825: connect: Connection refused (error 111)
I20260808 07:06:01.987887 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 32283
I20260808 07:06:01.995709 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 31895
2026-08-08T07:06:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2529 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T07:06:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:02Z Disabled control of system clock
I20260808 07:06:02.029157 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:46405
--webserver_interface=127.13.207.62
--webserver_port=0
--builtin_ntp_servers=127.13.207.20:41229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:46405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:06:02.127009 32467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:02.127296 32467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:02.127322 32467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:02.128713 32467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:02.128787 32467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:02.128801 32467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:02.128815 32467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:02.128829 32467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:02.130414 32467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:41229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:46405
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:46405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:02.130827 32467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:02.131258 32467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:02.137945 32473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:02.138028 32475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:02.137916 32472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:02.138782 32467 server_base.cc:1034] running on GCE node
I20260808 07:06:02.139132 32467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:02.139504 32467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:02.140703 32467 hybrid_clock.cc:648] HybridClock initialized: now 1786172762140674 us; error 42 us; skew 500 ppm
I20260808 07:06:02.143684 32467 webserver.cc:460] Webserver started at http://127.13.207.62:39021/ using document root <none> and password file <none>
I20260808 07:06:02.144184 32467 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:02.144380 32467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:02.144582 32467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:02.146024 32467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "b9c8c5faf5e644c980d5cf1a2854df5f"
format_stamp: "Formatted at 2026-08-08 07:06:02 on dist-test-slave-8tdl"
server_key: "1df86ee25209e3b7d3bc7664901527b1"
server_key_iv: "bf683e2d5357a4abd7dd504a76ed5317"
server_key_version: "encryptionkey@0"
I20260808 07:06:02.146900 32467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "b9c8c5faf5e644c980d5cf1a2854df5f"
format_stamp: "Formatted at 2026-08-08 07:06:02 on dist-test-slave-8tdl"
server_key: "1df86ee25209e3b7d3bc7664901527b1"
server_key_iv: "bf683e2d5357a4abd7dd504a76ed5317"
server_key_version: "encryptionkey@0"
I20260808 07:06:02.149235 32467 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 07:06:02.150489 32482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:02.150835 32467 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:02.150960 32467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "b9c8c5faf5e644c980d5cf1a2854df5f"
format_stamp: "Formatted at 2026-08-08 07:06:02 on dist-test-slave-8tdl"
server_key: "1df86ee25209e3b7d3bc7664901527b1"
server_key_iv: "bf683e2d5357a4abd7dd504a76ed5317"
server_key_version: "encryptionkey@0"
I20260808 07:06:02.151078 32467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:02.159987 32467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:02.160421 32467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:02.160629 32467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:02.166913 32467 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:46405
I20260808 07:06:02.167192 32536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:46405 every 8 connection(s)
I20260808 07:06:02.167485 32467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:02.168752 32537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:02.172091 32537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f: Bootstrap starting.
I20260808 07:06:02.172863 32537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:02.173242 32537 log.cc:824] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:02.174250 32537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f: No bootstrap required, opened a new log
I20260808 07:06:02.175987 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 32467
I20260808 07:06:02.176121 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:02.176333 14140 external_mini_cluster.cc:1428] Setting key 37d244c87823c99df9965c4eba3f0d9b
I20260808 07:06:02.177106 32537 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9c8c5faf5e644c980d5cf1a2854df5f" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 46405 } }
I20260808 07:06:02.177340 32537 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:02.177376 32537 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9c8c5faf5e644c980d5cf1a2854df5f, State: Initialized, Role: FOLLOWER
I20260808 07:06:02.177557 32537 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9c8c5faf5e644c980d5cf1a2854df5f" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 46405 } }
I20260808 07:06:02.177656 32537 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:02.177701 32537 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:02.177757 32537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:02.178500 32537 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9c8c5faf5e644c980d5cf1a2854df5f" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 46405 } }
I20260808 07:06:02.178645 32537 leader_election.cc:302] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9c8c5faf5e644c980d5cf1a2854df5f; no voters:
I20260808 07:06:02.178885 32537 leader_election.cc:288] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:02.178944 32542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:02.179236 32542 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f [term 1 LEADER]: Becoming Leader. State: Replica: b9c8c5faf5e644c980d5cf1a2854df5f, State: Running, Role: LEADER
I20260808 07:06:02.179373 32537 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:02.179435 32542 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9c8c5faf5e644c980d5cf1a2854df5f" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 46405 } }
I20260808 07:06:02.179883 32542 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9c8c5faf5e644c980d5cf1a2854df5f. Latest consensus state: current_term: 1 leader_uuid: "b9c8c5faf5e644c980d5cf1a2854df5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9c8c5faf5e644c980d5cf1a2854df5f" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 46405 } } }
I20260808 07:06:02.180166 32542 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:02.180087 32544 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9c8c5faf5e644c980d5cf1a2854df5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9c8c5faf5e644c980d5cf1a2854df5f" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 46405 } } }
I20260808 07:06:02.180984 32544 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f [sys.catalog]: This master's current role is: LEADER
W20260808 07:06:02.181624 32559 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:06:02.181706 32559 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:06:02.181772 32553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:02.182008 32553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:02.184321 32553 catalog_manager.cc:1347] Generated new cluster ID: af7efff09cd64b6b8bb9588017765da5
I20260808 07:06:02.184403 32553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:02.189014 32541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:02.177049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53486 (local address 127.13.207.62:46405)
0808 07:06:02.177312 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:02.177320 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:02.177374 (+ 54us) server_negotiation.cc:407] Connection header received
0808 07:06:02.177532 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:02.177542 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:02.177625 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:02.177794 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:02.178171 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.179911 (+ 1740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.181205 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.181897 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.182035 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:02.188103 (+ 6068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:02.188108 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:02.188109 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:02.188329 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:02.188385 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:02.188405 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:02.188499 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:02.188587 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 07:06:02.188692 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":128,"threads_started":1}
I20260808 07:06:02.195114 32553 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:02.195966 32553 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:02.204551 32553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f: Generated new TSK 0
I20260808 07:06:02.204805 32553 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:02.213814 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:46405
--builtin_ntp_servers=127.13.207.20:41229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:06:02.308992 32562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:02.309252 32562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:02.309280 32562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:02.310745 32562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:02.310802 32562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:02.310835 32562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:06:02.312448 32562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:41229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:46405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:02.312755 32562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:02.313130 32562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:02.318717 32570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:02.318717 32567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:02.318830 32568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:02.319633 32562 server_base.cc:1034] running on GCE node
I20260808 07:06:02.319967 32562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:02.320395 32562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:02.321614 32562 hybrid_clock.cc:648] HybridClock initialized: now 1786172762321592 us; error 43 us; skew 500 ppm
I20260808 07:06:02.323838 32562 webserver.cc:460] Webserver started at http://127.13.207.1:42695/ using document root <none> and password file <none>
I20260808 07:06:02.324199 32562 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:02.324285 32562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:02.324429 32562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:02.326032 32562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "ffd3288cba714abbb360fb189118f41f"
format_stamp: "Formatted at 2026-08-08 07:06:02 on dist-test-slave-8tdl"
server_key: "01c190c39ed8f5febfd187ab48e105cf"
server_key_iv: "839b9b4493f271edeba9c91dd13f7866"
server_key_version: "encryptionkey@0"
I20260808 07:06:02.326514 32562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "ffd3288cba714abbb360fb189118f41f"
format_stamp: "Formatted at 2026-08-08 07:06:02 on dist-test-slave-8tdl"
server_key: "01c190c39ed8f5febfd187ab48e105cf"
server_key_iv: "839b9b4493f271edeba9c91dd13f7866"
server_key_version: "encryptionkey@0"
I20260808 07:06:02.328462 32562 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260808 07:06:02.329576 32576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:02.329839 32562 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:02.329947 32562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "ffd3288cba714abbb360fb189118f41f"
format_stamp: "Formatted at 2026-08-08 07:06:02 on dist-test-slave-8tdl"
server_key: "01c190c39ed8f5febfd187ab48e105cf"
server_key_iv: "839b9b4493f271edeba9c91dd13f7866"
server_key_version: "encryptionkey@0"
I20260808 07:06:02.330034 32562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:02.356321 32562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:02.356662 32562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:02.356787 32562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:02.357016 32562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:02.357424 32562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:02.357465 32562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:02.357491 32562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:02.357506 32562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:02.365063 32562 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:43373
I20260808 07:06:02.365159 32698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:43373 every 8 connection(s)
I20260808 07:06:02.365584 32562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:06:02.371543 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 32562
I20260808 07:06:02.371670 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:06:02.371968 14140 external_mini_cluster.cc:1428] Setting key 2bebbae9b4f2dfd495fbad8162cb2fe5
I20260808 07:06:02.373046 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:46405
--builtin_ntp_servers=127.13.207.20:41229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 07:06:02.373157 32541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:02.366739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:52081 (local address 127.13.207.62:46405)
0808 07:06:02.366816 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:02.366819 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:02.367101 (+ 282us) server_negotiation.cc:407] Connection header received
0808 07:06:02.367692 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:02.367694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:02.367709 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:02.367785 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:02.368792 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.369452 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.370836 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.371116 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.371335 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:02.372781 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:02.372782 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:02.372782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:02.372788 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:02.372811 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:02.372816 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:02.372899 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:02.373015 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 07:06:02.373065 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:06:02.373445 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:02.366601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:46405 (local address 127.13.207.1:52081)
0808 07:06:02.366941 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:02.366991 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:02.367557 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:02.367985 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:02.367989 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:02.367999 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:02.368649 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:02.368655 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.369574 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.369575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:02.370619 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:02.370623 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.371430 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.371432 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:02.371661 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:02.372591 (+ 930us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:06:02.372591 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:02.372644 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:06:02.372682 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:02.372907 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:02.372916 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:02.372919 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:02.373020 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 07:06:02.373160 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":163,"threads_started":1}
I20260808 07:06:02.374285 32699 heartbeater.cc:347] Connected to a master server at 127.13.207.62:46405
I20260808 07:06:02.374418 32699 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:02.374715 32699 heartbeater.cc:511] Master 127.13.207.62:46405 requested a full tablet report, sending...
I20260808 07:06:02.375341 32499 ts_manager.cc:194] Registered new tserver with Master: ffd3288cba714abbb360fb189118f41f (127.13.207.1:43373)
I20260808 07:06:02.376012 32499 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.207.1:52081
W20260808 07:06:02.469260 32702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:02.469842 32702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:02.469944 32702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:02.472188 32702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:02.472409 32702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:02.472477 32702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:06:02.474434 32702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:41229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:46405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:02.474844 32702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:02.475267 32702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:02.482098 32710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:02.482177 32707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:02.482163 32708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:02.482502 32702 server_base.cc:1034] running on GCE node
I20260808 07:06:02.482887 32702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:02.483251 32702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:02.484444 32702 hybrid_clock.cc:648] HybridClock initialized: now 1786172762484402 us; error 58 us; skew 500 ppm
I20260808 07:06:02.485958 32702 webserver.cc:460] Webserver started at http://127.13.207.2:46387/ using document root <none> and password file <none>
I20260808 07:06:02.486322 32702 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:02.486454 32702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:02.486627 32702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:02.487866 32702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "4ada3a599d3f46e3a0cdcdf5df020788"
format_stamp: "Formatted at 2026-08-08 07:06:02 on dist-test-slave-8tdl"
server_key: "b20f45df16ed440c7db91a68872e0824"
server_key_iv: "d7cce9090fb27873b5ecf1ac5f282e77"
server_key_version: "encryptionkey@0"
I20260808 07:06:02.488543 32702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "4ada3a599d3f46e3a0cdcdf5df020788"
format_stamp: "Formatted at 2026-08-08 07:06:02 on dist-test-slave-8tdl"
server_key: "b20f45df16ed440c7db91a68872e0824"
server_key_iv: "d7cce9090fb27873b5ecf1ac5f282e77"
server_key_version: "encryptionkey@0"
I20260808 07:06:02.490283 32702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260808 07:06:02.491408 32716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:02.491814 32702 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:02.492034 32702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "4ada3a599d3f46e3a0cdcdf5df020788"
format_stamp: "Formatted at 2026-08-08 07:06:02 on dist-test-slave-8tdl"
server_key: "b20f45df16ed440c7db91a68872e0824"
server_key_iv: "d7cce9090fb27873b5ecf1ac5f282e77"
server_key_version: "encryptionkey@0"
I20260808 07:06:02.492139 32702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:02.512954 32702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:02.513383 32702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:02.513671 32702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:02.513974 32702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:02.514456 32702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:02.514518 32702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:02.514552 32702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:02.514570 32702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:02.521822 32702 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:42681
I20260808 07:06:02.521869 369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:42681 every 8 connection(s)
I20260808 07:06:02.522363 32702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:06:02.528434 32541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:02.523392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:33091 (local address 127.13.207.62:46405)
0808 07:06:02.523524 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:02.523528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:02.523722 (+ 194us) server_negotiation.cc:407] Connection header received
0808 07:06:02.524326 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:02.524327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:02.524341 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:02.524400 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:02.525154 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.525677 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.526817 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.527034 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.527115 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:02.528116 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:02.528117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:02.528118 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:02.528125 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:02.528142 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:02.528147 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:02.528215 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:02.528302 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 07:06:02.528339 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 07:06:02.528713 372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:02.523237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:46405 (local address 127.13.207.2:33091)
0808 07:06:02.523589 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:02.523635 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:02.524210 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:02.524481 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:02.524485 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:02.524500 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:02.525056 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:02.525062 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.525794 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.525795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:02.526718 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:02.526722 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.527130 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.527131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:02.527297 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:02.527934 (+ 637us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:06:02.527935 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:02.527967 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:06:02.527996 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:02.528216 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:02.528224 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:02.528228 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:02.528302 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 07:06:02.528437 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":106,"threads_started":1}
I20260808 07:06:02.529661 370 heartbeater.cc:347] Connected to a master server at 127.13.207.62:46405
I20260808 07:06:02.529775 370 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:02.529959 370 heartbeater.cc:511] Master 127.13.207.62:46405 requested a full tablet report, sending...
I20260808 07:06:02.530387 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 32702
I20260808 07:06:02.530470 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:06:02.530733 14140 external_mini_cluster.cc:1428] Setting key 98256ff53cc76e2657933042ad04220e
I20260808 07:06:02.530849 32499 ts_manager.cc:194] Registered new tserver with Master: 4ada3a599d3f46e3a0cdcdf5df020788 (127.13.207.2:42681)
I20260808 07:06:02.531344 32499 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.207.2:33091
I20260808 07:06:02.532117 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:46405
--builtin_ntp_servers=127.13.207.20:41229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:06:02.637565 373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:02.637914 373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:02.637967 373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:02.639567 373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:02.639652 373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:02.639705 373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:06:02.641525 373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:41229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:46405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:02.641903 373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:02.642305 373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:02.648959 381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:02.649081 379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:02.648911 378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:02.649185 373 server_base.cc:1034] running on GCE node
I20260808 07:06:02.649658 373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:02.649961 373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:02.651160 373 hybrid_clock.cc:648] HybridClock initialized: now 1786172762651099 us; error 80 us; skew 500 ppm
I20260808 07:06:02.652825 373 webserver.cc:460] Webserver started at http://127.13.207.3:42481/ using document root <none> and password file <none>
I20260808 07:06:02.653172 373 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:02.653258 373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:02.653421 373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:02.654881 373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "59f5b1abc7384816ab83700e3bc344de"
format_stamp: "Formatted at 2026-08-08 07:06:02 on dist-test-slave-8tdl"
server_key: "f7b5839ae6c9cb40645ac84bb16bdfb5"
server_key_iv: "663cac8d6e6aa6fa5369848d33869211"
server_key_version: "encryptionkey@0"
I20260808 07:06:02.655351 373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "59f5b1abc7384816ab83700e3bc344de"
format_stamp: "Formatted at 2026-08-08 07:06:02 on dist-test-slave-8tdl"
server_key: "f7b5839ae6c9cb40645ac84bb16bdfb5"
server_key_iv: "663cac8d6e6aa6fa5369848d33869211"
server_key_version: "encryptionkey@0"
I20260808 07:06:02.656973 373 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:06:02.658172 387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:02.658443 373 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:02.658567 373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "59f5b1abc7384816ab83700e3bc344de"
format_stamp: "Formatted at 2026-08-08 07:06:02 on dist-test-slave-8tdl"
server_key: "f7b5839ae6c9cb40645ac84bb16bdfb5"
server_key_iv: "663cac8d6e6aa6fa5369848d33869211"
server_key_version: "encryptionkey@0"
I20260808 07:06:02.658672 373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:02.682015 373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:02.682410 373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:02.682583 373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:02.682876 373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:02.683260 373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:02.683316 373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:02.683363 373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:02.683393 373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:02.692808 373 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:42605
I20260808 07:06:02.692899 504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:42605 every 8 connection(s)
I20260808 07:06:02.693324 373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:06:02.700459 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 373
I20260808 07:06:02.700587 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:06:02.700578 32541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:02.694385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:58011 (local address 127.13.207.62:46405)
0808 07:06:02.694482 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:02.694484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:02.695371 (+ 887us) server_negotiation.cc:407] Connection header received
0808 07:06:02.696084 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:02.696086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:02.696102 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:02.696161 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:02.696954 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.697529 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.698733 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.698967 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.699063 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:02.700138 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:02.700140 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:02.700140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:02.700147 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:02.700172 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:02.700177 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:02.700246 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:02.700356 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 07:06:02.700398 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 07:06:02.700788 508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:02.694591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:46405 (local address 127.13.207.3:58011)
0808 07:06:02.695086 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:02.695178 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:02.695939 (+ 761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:02.696250 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:02.696254 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:02.696262 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:02.696851 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:02.696856 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.697661 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.697662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:02.698588 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:02.698591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.699082 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.699083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:02.699244 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:02.699866 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:06:02.699867 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:02.699904 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:06:02.699944 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:02.700248 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:02.700257 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:02.700261 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:02.700361 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 07:06:02.700499 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":215,"threads_started":1}
I20260808 07:06:02.700807 14140 external_mini_cluster.cc:1428] Setting key dd9fa9b0cce3e16a4e70e2619b41f59f
I20260808 07:06:02.701642 506 heartbeater.cc:347] Connected to a master server at 127.13.207.62:46405
I20260808 07:06:02.701785 506 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:02.702054 506 heartbeater.cc:511] Master 127.13.207.62:46405 requested a full tablet report, sending...
I20260808 07:06:02.702715 32499 ts_manager.cc:194] Registered new tserver with Master: 59f5b1abc7384816ab83700e3bc344de (127.13.207.3:42605)
I20260808 07:06:02.703116 32499 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.207.3:58011
I20260808 07:06:02.714074 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:06:02.719314 32541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:02.715872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53502 (local address 127.13.207.62:46405)
0808 07:06:02.715929 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:02.715932 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:02.715987 (+ 55us) server_negotiation.cc:407] Connection header received
0808 07:06:02.716137 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:02.716138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:02.716154 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:02.716217 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:02.716495 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.717015 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.717888 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.718094 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.718159 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:02.719040 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:02.719041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:02.719042 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:02.719048 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:02.719064 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:02.719068 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:02.719165 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:02.719211 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 07:06:02.719243 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:06:02.721632 32499 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:06:02.723882 32499 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:02.731971 517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:02.727277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:43373 (local address 127.0.0.1:50850)
0808 07:06:02.727439 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:02.727477 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:02.727617 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:02.728140 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:02.728144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:02.728147 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:02.728240 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:02.728243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.729466 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.729467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:02.730635 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:02.730639 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.731631 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.731632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:02.731729 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:02.731745 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 07:06:02.731823 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:06:02.731883 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":110,"threads_started":1}
I20260808 07:06:02.731983 518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:02.727421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50850 (local address 127.13.207.1:43373)
0808 07:06:02.727662 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:02.727668 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:02.727681 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:06:02.727808 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:02.727811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:02.727830 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:02.727925 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:02.728323 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.729373 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.730774 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.731558 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.731630 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:02.731735 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:02.731839 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 07:06:02.731879 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":116,"threads_started":1}
I20260808 07:06:02.732710 521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:02.727814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:42681 (local address 127.0.0.1:53060)
0808 07:06:02.728301 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:02.728310 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:02.728359 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:02.728741 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:02.728742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:02.728744 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:02.728790 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:02.728791 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.730162 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.730163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:02.731195 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:02.731198 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.732428 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.732429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:02.732500 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:02.732506 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:06:02.732565 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 07:06:02.732618 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":32,"thread_start_us":71,"threads_started":1}
I20260808 07:06:02.732913 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:02.727744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59766 (local address 127.13.207.3:42605)
0808 07:06:02.727983 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:02.727988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:02.728173 (+ 185us) server_negotiation.cc:407] Connection header received
0808 07:06:02.728225 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:02.728228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:02.728245 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:02.728325 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:02.728704 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.730255 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.731547 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.732322 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.732396 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:02.732485 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:02.732762 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 07:06:02.732820 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":101,"threads_started":1}
I20260808 07:06:02.733031 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:02.727955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53060 (local address 127.13.207.2:42681)
0808 07:06:02.728204 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:02.728209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:02.728366 (+ 157us) server_negotiation.cc:407] Connection header received
0808 07:06:02.728549 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:02.728552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:02.728573 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:02.728661 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:02.728880 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.730050 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.731348 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.732196 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.732283 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:02.732381 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:02.732859 (+ 478us) server_negotiation.cc:299] Negotiation successful
0808 07:06:02.732912 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":110,"threads_started":1}
I20260808 07:06:02.733095 519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:02.727603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:42605 (local address 127.0.0.1:59766)
0808 07:06:02.728046 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:02.728056 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:02.728112 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:02.728463 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:02.728464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:02.728467 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:02.728531 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:02.728533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.730340 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.730341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:02.731444 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:02.731447 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:02.732536 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:02.732537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:02.732600 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:02.732603 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:02.732653 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 07:06:02.732690 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":17,"thread_start_us":90,"threads_started":1}
I20260808 07:06:02.733278 32625 tablet_service.cc:1467] Processing CreateTablet for tablet 7f021a8973164a458aadaa4bfd006c45 (DEFAULT_TABLE table=test-table [id=a8863b0377554409bf54c5a208f975b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:02.733722 32625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f021a8973164a458aadaa4bfd006c45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:02.733682 300 tablet_service.cc:1467] Processing CreateTablet for tablet 7f021a8973164a458aadaa4bfd006c45 (DEFAULT_TABLE table=test-table [id=a8863b0377554409bf54c5a208f975b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:02.734023 300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f021a8973164a458aadaa4bfd006c45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:02.733908 437 tablet_service.cc:1467] Processing CreateTablet for tablet 7f021a8973164a458aadaa4bfd006c45 (DEFAULT_TABLE table=test-table [id=a8863b0377554409bf54c5a208f975b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:02.734313 437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f021a8973164a458aadaa4bfd006c45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:02.737560 523 tablet_bootstrap.cc:492] T 7f021a8973164a458aadaa4bfd006c45 P 59f5b1abc7384816ab83700e3bc344de: Bootstrap starting.
I20260808 07:06:02.737869 524 tablet_bootstrap.cc:492] T 7f021a8973164a458aadaa4bfd006c45 P 4ada3a599d3f46e3a0cdcdf5df020788: Bootstrap starting.
I20260808 07:06:02.738461 523 tablet_bootstrap.cc:654] T 7f021a8973164a458aadaa4bfd006c45 P 59f5b1abc7384816ab83700e3bc344de: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:02.738623 524 tablet_bootstrap.cc:654] T 7f021a8973164a458aadaa4bfd006c45 P 4ada3a599d3f46e3a0cdcdf5df020788: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:02.738845 523 log.cc:824] T 7f021a8973164a458aadaa4bfd006c45 P 59f5b1abc7384816ab83700e3bc344de: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:02.739013 524 log.cc:824] T 7f021a8973164a458aadaa4bfd006c45 P 4ada3a599d3f46e3a0cdcdf5df020788: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:02.739286 525 tablet_bootstrap.cc:492] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f: Bootstrap starting.
I20260808 07:06:02.739847 523 tablet_bootstrap.cc:492] T 7f021a8973164a458aadaa4bfd006c45 P 59f5b1abc7384816ab83700e3bc344de: No bootstrap required, opened a new log
I20260808 07:06:02.739924 524 tablet_bootstrap.cc:492] T 7f021a8973164a458aadaa4bfd006c45 P 4ada3a599d3f46e3a0cdcdf5df020788: No bootstrap required, opened a new log
I20260808 07:06:02.739945 523 ts_tablet_manager.cc:1397] T 7f021a8973164a458aadaa4bfd006c45 P 59f5b1abc7384816ab83700e3bc344de: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:06:02.740051 524 ts_tablet_manager.cc:1397] T 7f021a8973164a458aadaa4bfd006c45 P 4ada3a599d3f46e3a0cdcdf5df020788: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:06:02.740141 525 tablet_bootstrap.cc:654] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:02.740459 525 log.cc:824] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:02.741235 525 tablet_bootstrap.cc:492] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f: No bootstrap required, opened a new log
I20260808 07:06:02.741328 525 ts_tablet_manager.cc:1397] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:06:02.743093 523 raft_consensus.cc:348] T 7f021a8973164a458aadaa4bfd006c45 P 59f5b1abc7384816ab83700e3bc344de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd3288cba714abbb360fb189118f41f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 43373 } } peers { permanent_uuid: "59f5b1abc7384816ab83700e3bc344de" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 42605 } } peers { permanent_uuid: "4ada3a599d3f46e3a0cdcdf5df020788" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 42681 } }
I20260808 07:06:02.743224 524 raft_consensus.cc:348] T 7f021a8973164a458aadaa4bfd006c45 P 4ada3a599d3f46e3a0cdcdf5df020788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd3288cba714abbb360fb189118f41f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 43373 } } peers { permanent_uuid: "59f5b1abc7384816ab83700e3bc344de" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 42605 } } peers { permanent_uuid: "4ada3a599d3f46e3a0cdcdf5df020788" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 42681 } }
I20260808 07:06:02.743319 523 raft_consensus.cc:374] T 7f021a8973164a458aadaa4bfd006c45 P 59f5b1abc7384816ab83700e3bc344de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:02.743361 524 raft_consensus.cc:374] T 7f021a8973164a458aadaa4bfd006c45 P 4ada3a599d3f46e3a0cdcdf5df020788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:02.743359 523 raft_consensus.cc:729] T 7f021a8973164a458aadaa4bfd006c45 P 59f5b1abc7384816ab83700e3bc344de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59f5b1abc7384816ab83700e3bc344de, State: Initialized, Role: FOLLOWER
I20260808 07:06:02.743436 524 raft_consensus.cc:729] T 7f021a8973164a458aadaa4bfd006c45 P 4ada3a599d3f46e3a0cdcdf5df020788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ada3a599d3f46e3a0cdcdf5df020788, State: Initialized, Role: FOLLOWER
I20260808 07:06:02.743532 523 consensus_queue.cc:260] T 7f021a8973164a458aadaa4bfd006c45 P 59f5b1abc7384816ab83700e3bc344de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd3288cba714abbb360fb189118f41f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 43373 } } peers { permanent_uuid: "59f5b1abc7384816ab83700e3bc344de" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 42605 } } peers { permanent_uuid: "4ada3a599d3f46e3a0cdcdf5df020788" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 42681 } }
I20260808 07:06:02.743580 524 consensus_queue.cc:260] T 7f021a8973164a458aadaa4bfd006c45 P 4ada3a599d3f46e3a0cdcdf5df020788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd3288cba714abbb360fb189118f41f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 43373 } } peers { permanent_uuid: "59f5b1abc7384816ab83700e3bc344de" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 42605 } } peers { permanent_uuid: "4ada3a599d3f46e3a0cdcdf5df020788" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 42681 } }
I20260808 07:06:02.743884 524 ts_tablet_manager.cc:1428] T 7f021a8973164a458aadaa4bfd006c45 P 4ada3a599d3f46e3a0cdcdf5df020788: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 07:06:02.743885 523 ts_tablet_manager.cc:1428] T 7f021a8973164a458aadaa4bfd006c45 P 59f5b1abc7384816ab83700e3bc344de: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 07:06:02.743954 506 heartbeater.cc:503] Master 127.13.207.62:46405 was elected leader, sending a full tablet report...
I20260808 07:06:02.743924 525 raft_consensus.cc:348] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd3288cba714abbb360fb189118f41f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 43373 } } peers { permanent_uuid: "59f5b1abc7384816ab83700e3bc344de" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 42605 } } peers { permanent_uuid: "4ada3a599d3f46e3a0cdcdf5df020788" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 42681 } }
I20260808 07:06:02.744130 525 raft_consensus.cc:374] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:02.744169 525 raft_consensus.cc:729] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffd3288cba714abbb360fb189118f41f, State: Initialized, Role: FOLLOWER
I20260808 07:06:02.744185 370 heartbeater.cc:503] Master 127.13.207.62:46405 was elected leader, sending a full tablet report...
I20260808 07:06:02.744355 525 consensus_queue.cc:260] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd3288cba714abbb360fb189118f41f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 43373 } } peers { permanent_uuid: "59f5b1abc7384816ab83700e3bc344de" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 42605 } } peers { permanent_uuid: "4ada3a599d3f46e3a0cdcdf5df020788" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 42681 } }
I20260808 07:06:02.744702 525 ts_tablet_manager.cc:1428] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 07:06:02.744836 32699 heartbeater.cc:503] Master 127.13.207.62:46405 was elected leader, sending a full tablet report...
W20260808 07:06:02.773288 371 tablet.cc:2369] T 7f021a8973164a458aadaa4bfd006c45 P 4ada3a599d3f46e3a0cdcdf5df020788: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:06:02.866496 32700 tablet.cc:2369] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:06:02.944128 507 tablet.cc:2369] T 7f021a8973164a458aadaa4bfd006c45 P 59f5b1abc7384816ab83700e3bc344de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:06:03.115706 531 raft_consensus.cc:482] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:03.116039 531 raft_consensus.cc:504] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd3288cba714abbb360fb189118f41f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 43373 } } peers { permanent_uuid: "59f5b1abc7384816ab83700e3bc344de" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 42605 } } peers { permanent_uuid: "4ada3a599d3f46e3a0cdcdf5df020788" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 42681 } }
I20260808 07:06:03.116720 531 leader_election.cc:288] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59f5b1abc7384816ab83700e3bc344de (127.13.207.3:42605), 4ada3a599d3f46e3a0cdcdf5df020788 (127.13.207.2:42681)
I20260808 07:06:03.120174 533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:03.117290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:42681 (local address 127.13.207.1:32833)
0808 07:06:03.117386 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:03.117402 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:03.117450 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:03.117645 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:03.117646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:03.117648 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:03.117731 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:03.117733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:03.118401 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:03.118403 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:03.119266 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:03.119269 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:03.119885 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:03.119886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:03.119961 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:03.119964 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:03.120030 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 07:06:03.120075 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":43,"threads_started":1}
I20260808 07:06:03.120159 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:03.117444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:32833 (local address 127.13.207.2:42681)
0808 07:06:03.117519 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:03.117522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:03.117531 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:06:03.117560 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:03.117561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:03.117577 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:03.117641 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:03.117803 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:03.118286 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:03.119371 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:03.119787 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:03.119884 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:03.119895 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:03.120037 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 07:06:03.120079 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 07:06:03.120651 322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f021a8973164a458aadaa4bfd006c45" candidate_uuid: "ffd3288cba714abbb360fb189118f41f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ada3a599d3f46e3a0cdcdf5df020788" is_pre_election: true
I20260808 07:06:03.120834 532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:03.117067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:42605 (local address 127.13.207.1:35815)
0808 07:06:03.117256 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:03.117272 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:03.117426 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:03.117728 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:03.117729 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:03.117731 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:03.117880 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:03.117884 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:03.118875 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:03.118876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:03.119815 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:03.119818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:03.120616 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:03.120617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:03.120671 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:03.120674 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:03.120720 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 07:06:03.120750 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":92,"threads_started":1}
I20260808 07:06:03.120882 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:03.117175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:35815 (local address 127.13.207.3:42605)
0808 07:06:03.117266 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:03.117269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:03.117382 (+ 113us) server_negotiation.cc:407] Connection header received
0808 07:06:03.117539 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:03.117540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:03.117575 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:03.117715 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:03.118168 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:03.118775 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:03.119938 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:03.120508 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:03.120601 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:03.120610 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:03.120754 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 07:06:03.120810 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 07:06:03.120888 322 raft_consensus.cc:2454] T 7f021a8973164a458aadaa4bfd006c45 P 4ada3a599d3f46e3a0cdcdf5df020788 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffd3288cba714abbb360fb189118f41f in term 0.
I20260808 07:06:03.121181 32580 leader_election.cc:302] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ada3a599d3f46e3a0cdcdf5df020788, ffd3288cba714abbb360fb189118f41f; no voters:
I20260808 07:06:03.121304 458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f021a8973164a458aadaa4bfd006c45" candidate_uuid: "ffd3288cba714abbb360fb189118f41f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59f5b1abc7384816ab83700e3bc344de" is_pre_election: true
I20260808 07:06:03.121405 531 raft_consensus.cc:2781] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:03.121461 458 raft_consensus.cc:2454] T 7f021a8973164a458aadaa4bfd006c45 P 59f5b1abc7384816ab83700e3bc344de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffd3288cba714abbb360fb189118f41f in term 0.
I20260808 07:06:03.121495 531 raft_consensus.cc:482] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:03.121517 531 raft_consensus.cc:3037] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:03.122419 531 raft_consensus.cc:504] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd3288cba714abbb360fb189118f41f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 43373 } } peers { permanent_uuid: "59f5b1abc7384816ab83700e3bc344de" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 42605 } } peers { permanent_uuid: "4ada3a599d3f46e3a0cdcdf5df020788" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 42681 } }
I20260808 07:06:03.122558 531 leader_election.cc:288] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f [CANDIDATE]: Term 1 election: Requested vote from peers 59f5b1abc7384816ab83700e3bc344de (127.13.207.3:42605), 4ada3a599d3f46e3a0cdcdf5df020788 (127.13.207.2:42681)
I20260808 07:06:03.122825 458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f021a8973164a458aadaa4bfd006c45" candidate_uuid: "ffd3288cba714abbb360fb189118f41f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59f5b1abc7384816ab83700e3bc344de"
I20260808 07:06:03.122926 458 raft_consensus.cc:3037] T 7f021a8973164a458aadaa4bfd006c45 P 59f5b1abc7384816ab83700e3bc344de [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:03.122984 322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f021a8973164a458aadaa4bfd006c45" candidate_uuid: "ffd3288cba714abbb360fb189118f41f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ada3a599d3f46e3a0cdcdf5df020788"
I20260808 07:06:03.123184 322 raft_consensus.cc:3037] T 7f021a8973164a458aadaa4bfd006c45 P 4ada3a599d3f46e3a0cdcdf5df020788 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:03.123813 458 raft_consensus.cc:2454] T 7f021a8973164a458aadaa4bfd006c45 P 59f5b1abc7384816ab83700e3bc344de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffd3288cba714abbb360fb189118f41f in term 1.
I20260808 07:06:03.124022 32580 leader_election.cc:302] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59f5b1abc7384816ab83700e3bc344de, ffd3288cba714abbb360fb189118f41f; no voters:
I20260808 07:06:03.124199 531 raft_consensus.cc:2781] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:03.124194 322 raft_consensus.cc:2454] T 7f021a8973164a458aadaa4bfd006c45 P 4ada3a599d3f46e3a0cdcdf5df020788 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffd3288cba714abbb360fb189118f41f in term 1.
I20260808 07:06:03.124451 531 raft_consensus.cc:686] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f [term 1 LEADER]: Becoming Leader. State: Replica: ffd3288cba714abbb360fb189118f41f, State: Running, Role: LEADER
I20260808 07:06:03.124603 531 consensus_queue.cc:237] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd3288cba714abbb360fb189118f41f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 43373 } } peers { permanent_uuid: "59f5b1abc7384816ab83700e3bc344de" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 42605 } } peers { permanent_uuid: "4ada3a599d3f46e3a0cdcdf5df020788" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 42681 } }
I20260808 07:06:03.125751 32496 catalog_manager.cc:5495] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f reported cstate change: term changed from 0 to 1, leader changed from <none> to ffd3288cba714abbb360fb189118f41f (127.13.207.1). New cstate: current_term: 1 leader_uuid: "ffd3288cba714abbb360fb189118f41f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd3288cba714abbb360fb189118f41f" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 43373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59f5b1abc7384816ab83700e3bc344de" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 42605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ada3a599d3f46e3a0cdcdf5df020788" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 42681 } health_report { overall_health: UNKNOWN } } }
I20260808 07:06:03.182616 518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:03.178945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50856 (local address 127.13.207.1:43373)
0808 07:06:03.179016 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:03.179018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:03.179116 (+ 98us) server_negotiation.cc:407] Connection header received
0808 07:06:03.179202 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:03.179204 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:03.179233 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:03.179305 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:03.179545 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:03.180309 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:03.181388 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:03.181644 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:03.181716 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:03.182074 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:03.182203 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:03.182294 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:03.182437 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 07:06:03.182494 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:06:03.185098 322 raft_consensus.cc:1261] T 7f021a8973164a458aadaa4bfd006c45 P 4ada3a599d3f46e3a0cdcdf5df020788 [term 1 FOLLOWER]: Refusing update from remote peer ffd3288cba714abbb360fb189118f41f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:03.185141 458 raft_consensus.cc:1261] T 7f021a8973164a458aadaa4bfd006c45 P 59f5b1abc7384816ab83700e3bc344de [term 1 FOLLOWER]: Refusing update from remote peer ffd3288cba714abbb360fb189118f41f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:03.185591 531 consensus_queue.cc:1035] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ada3a599d3f46e3a0cdcdf5df020788" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 42681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:03.185920 534 consensus_queue.cc:1035] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f [LEADER]: Connected to new peer: Peer: permanent_uuid: "59f5b1abc7384816ab83700e3bc344de" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 42605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:03.188834 537 mvcc.cc:205] Tried to move back new op lower bound from 7316163638002253824 to 7316163637759201280. Current Snapshot: MvccSnapshot[applied={T|T < 7316163638002253824}]
I20260808 07:06:03.189270 540 mvcc.cc:205] Tried to move back new op lower bound from 7316163638002253824 to 7316163637759201280. Current Snapshot: MvccSnapshot[applied={T|T < 7316163638002253824}]
I20260808 07:06:03.189720 539 mvcc.cc:205] Tried to move back new op lower bound from 7316163638002253824 to 7316163637759201280. Current Snapshot: MvccSnapshot[applied={T|T < 7316163638002253824}]
I20260808 07:06:03.192683 32496 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:03.192806 32496 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:03.193821 32496 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b9c8c5faf5e644c980d5cf1a2854df5f: Sending DeleteTablet for 3 replicas of tablet 7f021a8973164a458aadaa4bfd006c45
I20260808 07:06:03.194278 32625 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f021a8973164a458aadaa4bfd006c45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:03 UTC) from {username='slave'} at 127.0.0.1:50850
I20260808 07:06:03.194340 300 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f021a8973164a458aadaa4bfd006c45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:03 UTC) from {username='slave'} at 127.0.0.1:53060
I20260808 07:06:03.194624 548 tablet_replica.cc:330] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f: stopping tablet replica
I20260808 07:06:03.194620 437 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f021a8973164a458aadaa4bfd006c45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:03 UTC) from {username='slave'} at 127.0.0.1:59766
I20260808 07:06:03.194829 548 raft_consensus.cc:2229] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:06:03.194983 548 raft_consensus.cc:2258] T 7f021a8973164a458aadaa4bfd006c45 P ffd3288cba714abbb360fb189118f41f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:03.195181 550 tablet_replica.cc:330] T 7f021a8973164a458aadaa4bfd006c45 P 59f5b1abc7384816ab83700e3bc344de: stopping tablet replica
I20260808 07:06:03.195257 549 tablet_replica.cc:330] T 7f021a8973164a458aadaa4bfd006c45 P 4ada3a599d3f46e3a0cdcdf5df020788: stopping tablet replica
I20260808 07:06:03.195304 550 raft_consensus.cc:2229] T 7f021a8973164a458aadaa4bfd006c45 P 59f5b1abc7384816ab83700e3bc344de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:03.195369 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 32562
I20260808 07:06:03.195421 550 raft_consensus.cc:2258] T 7f021a8973164a458aadaa4bfd006c45 P 59f5b1abc7384816ab83700e3bc344de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:03.195525 549 raft_consensus.cc:2229] T 7f021a8973164a458aadaa4bfd006c45 P 4ada3a599d3f46e3a0cdcdf5df020788 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:03.195771 549 raft_consensus.cc:2258] T 7f021a8973164a458aadaa4bfd006c45 P 4ada3a599d3f46e3a0cdcdf5df020788 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:03.197284 550 ts_tablet_manager.cc:1905] T 7f021a8973164a458aadaa4bfd006c45 P 59f5b1abc7384816ab83700e3bc344de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:03.200021 549 ts_tablet_manager.cc:1905] T 7f021a8973164a458aadaa4bfd006c45 P 4ada3a599d3f46e3a0cdcdf5df020788: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:03.200428 550 ts_tablet_manager.cc:1918] T 7f021a8973164a458aadaa4bfd006c45 P 59f5b1abc7384816ab83700e3bc344de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:03.200541 550 log.cc:1196] T 7f021a8973164a458aadaa4bfd006c45 P 59f5b1abc7384816ab83700e3bc344de: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/7f021a8973164a458aadaa4bfd006c45
I20260808 07:06:03.200857 550 ts_tablet_manager.cc:1939] T 7f021a8973164a458aadaa4bfd006c45 P 59f5b1abc7384816ab83700e3bc344de: Deleting consensus metadata
I20260808 07:06:03.201644 32486 catalog_manager.cc:4841] TS 59f5b1abc7384816ab83700e3bc344de (127.13.207.3:42605): tablet 7f021a8973164a458aadaa4bfd006c45 (table test-table [id=a8863b0377554409bf54c5a208f975b8]) successfully deleted
I20260808 07:06:03.202243 549 ts_tablet_manager.cc:1918] T 7f021a8973164a458aadaa4bfd006c45 P 4ada3a599d3f46e3a0cdcdf5df020788: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:03.202348 549 log.cc:1196] T 7f021a8973164a458aadaa4bfd006c45 P 4ada3a599d3f46e3a0cdcdf5df020788: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172493058068-14140-0/minicluster-data/ts-1/wal/wals/7f021a8973164a458aadaa4bfd006c45
I20260808 07:06:03.202636 549 ts_tablet_manager.cc:1939] T 7f021a8973164a458aadaa4bfd006c45 P 4ada3a599d3f46e3a0cdcdf5df020788: Deleting consensus metadata
I20260808 07:06:03.203774 32486 catalog_manager.cc:4841] TS 4ada3a599d3f46e3a0cdcdf5df020788 (127.13.207.2:42681): tablet 7f021a8973164a458aadaa4bfd006c45 (table test-table [id=a8863b0377554409bf54c5a208f975b8]) successfully deleted
W20260808 07:06:03.205262 32486 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.207.1:43373 (error 108)
I20260808 07:06:03.206082 519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:03.205790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:43373 (local address 127.0.0.1:50870)
0808 07:06:03.205875 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:03.206039 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.1:43373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 07:06:03.206069 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 32702
W20260808 07:06:03.207594 32486 catalog_manager.cc:4584] TS ffd3288cba714abbb360fb189118f41f (127.13.207.1:43373): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7f021a8973164a458aadaa4bfd006c45: Network error: Client connection negotiation failed: client connection to 127.13.207.1:43373: connect: Connection refused (error 111)
I20260808 07:06:03.216262 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 373
I20260808 07:06:03.223928 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 32467
2026-08-08T07:06:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1226 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:03 dist-test-slave-8tdl krb5kdc[555](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:03 dist-test-slave-8tdl krb5kdc[555](info): set up 2 sockets
Aug 08 07:06:03 dist-test-slave-8tdl krb5kdc[555](info): commencing operation
krb5kdc: starting...
W20260808 07:06:04.372884 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:06:04 dist-test-slave-8tdl krb5kdc[555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:06:04.531273 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:40919
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:42999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:40919
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:04.627398 569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:04.627688 569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:04.627736 569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:04.629087 569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:04.629170 569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:04.629195 569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:04.629217 569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:04.629248 569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:04.630981 569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:42999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:40919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:40919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:04.631299 569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:04.631567 569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:04.637562 574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:04.637588 578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:04.637575 576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:04.638010 569 server_base.cc:1034] running on GCE node
I20260808 07:06:04.638388 569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:04.638864 569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:04.640153 569 hybrid_clock.cc:648] HybridClock initialized: now 1786172764640114 us; error 54 us; skew 500 ppm
Aug 08 07:06:04 dist-test-slave-8tdl krb5kdc[555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172764, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:04.643348 569 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:06:04.643945 569 webserver.cc:460] Webserver started at http://127.13.207.62:35113/ using document root <none> and password file <none>
I20260808 07:06:04.644205 569 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:04.644265 569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:04.644484 569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:04.645767 569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "38aecd6d834748099582ec389552ae68"
format_stamp: "Formatted at 2026-08-08 07:06:04 on dist-test-slave-8tdl"
server_key: "06c352968eb3408146ea5d668926af3c"
server_key_iv: "30dff8e3171ec4fcde37261654c0ef2d"
server_key_version: "encryptionkey@0"
I20260808 07:06:04.646186 569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "38aecd6d834748099582ec389552ae68"
format_stamp: "Formatted at 2026-08-08 07:06:04 on dist-test-slave-8tdl"
server_key: "06c352968eb3408146ea5d668926af3c"
server_key_iv: "30dff8e3171ec4fcde37261654c0ef2d"
server_key_version: "encryptionkey@0"
I20260808 07:06:04.648025 569 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 07:06:04.649062 587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:04.649472 569 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:04.649618 569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "38aecd6d834748099582ec389552ae68"
format_stamp: "Formatted at 2026-08-08 07:06:04 on dist-test-slave-8tdl"
server_key: "06c352968eb3408146ea5d668926af3c"
server_key_iv: "30dff8e3171ec4fcde37261654c0ef2d"
server_key_version: "encryptionkey@0"
I20260808 07:06:04.649709 569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:04.661695 569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:04.664522 569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:04.664804 569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:04.671007 569 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:40919
I20260808 07:06:04.671089 641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:40919 every 8 connection(s)
I20260808 07:06:04.671818 569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:04.673127 642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:04.676414 642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68: Bootstrap starting.
I20260808 07:06:04.677217 642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:04.677559 642 log.cc:824] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:04.678602 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 569
I20260808 07:06:04.678782 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:04.678999 642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68: No bootstrap required, opened a new log
I20260808 07:06:04.679037 14140 external_mini_cluster.cc:1428] Setting key 2ce978bca4996aab6cc0774ca30c8516
I20260808 07:06:04.681097 642 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38aecd6d834748099582ec389552ae68" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40919 } }
I20260808 07:06:04.681288 642 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:04.681334 642 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38aecd6d834748099582ec389552ae68, State: Initialized, Role: FOLLOWER
I20260808 07:06:04.681464 642 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38aecd6d834748099582ec389552ae68" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40919 } }
I20260808 07:06:04.681571 642 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:04.681619 642 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:04.681687 642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:04.682737 642 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38aecd6d834748099582ec389552ae68" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40919 } }
I20260808 07:06:04.682881 642 leader_election.cc:302] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38aecd6d834748099582ec389552ae68; no voters:
I20260808 07:06:04.683108 642 leader_election.cc:288] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:04.683154 647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:04.683301 647 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68 [term 1 LEADER]: Becoming Leader. State: Replica: 38aecd6d834748099582ec389552ae68, State: Running, Role: LEADER
I20260808 07:06:04.683470 647 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38aecd6d834748099582ec389552ae68" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40919 } }
I20260808 07:06:04.683480 642 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:04.684083 649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38aecd6d834748099582ec389552ae68. Latest consensus state: current_term: 1 leader_uuid: "38aecd6d834748099582ec389552ae68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38aecd6d834748099582ec389552ae68" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40919 } } }
I20260808 07:06:04.684381 649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:04.684057 648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38aecd6d834748099582ec389552ae68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38aecd6d834748099582ec389552ae68" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40919 } } }
I20260808 07:06:04.684649 648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:04.684792 651 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:04.685333 651 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:04.688006 651 catalog_manager.cc:1347] Generated new cluster ID: 7446469bc46442028b42e2393a9b0712
I20260808 07:06:04.688082 651 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:06:04 dist-test-slave-8tdl krb5kdc[555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:06:04.694579 646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:04.679610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50544 (local address 127.13.207.62:40919)
0808 07:06:04.679840 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:04.679848 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:04.679876 (+ 28us) server_negotiation.cc:407] Connection header received
0808 07:06:04.681130 (+ 1254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:04.681139 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:04.681212 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:04.681398 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:04.682236 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:04.684266 (+ 2030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:04.685638 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:04.686411 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:04.686543 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:04.690137 (+ 3594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:04.690142 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:04.690143 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:04.690159 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:04.692494 (+ 2335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:04.692963 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:04.692969 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:04.692969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:04.693164 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:04.693387 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:04.693388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:04.693389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:04.693763 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:04.693961 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:04.694051 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 07:06:04.694330 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":94,"threads_started":1}
I20260808 07:06:04.700762 651 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:04.701646 651 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:04.708451 651 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68: Generated new TSK 0
I20260808 07:06:04.708710 651 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
I20260808 07:06:04.759613 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40919
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:42999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:04.890486 670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:04.890888 670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:04.890940 670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:04.892625 670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:04.892695 670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:04.892729 670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:06:04.894618 670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:42999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:04.895012 670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:04.895300 670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:04.901266 677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:04.901413 670 server_base.cc:1034] running on GCE node
W20260808 07:06:04.901381 676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:04.901381 679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:04.901777 670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:04.902208 670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:04.903421 670 hybrid_clock.cc:648] HybridClock initialized: now 1786172764903412 us; error 62 us; skew 500 ppm
Aug 08 07:06:04 dist-test-slave-8tdl krb5kdc[555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172764, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:04.906375 670 init.cc:377] Logged in from keytab as kudu/127.13.207.1@KRBTEST.COM (short username kudu)
I20260808 07:06:04.907009 670 webserver.cc:460] Webserver started at http://127.13.207.1:39991/ using document root <none> and password file <none>
I20260808 07:06:04.907267 670 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:04.907339 670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:04.907486 670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:04.908701 670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "4900b6f337e24d1b9e05e81294fdce0b"
format_stamp: "Formatted at 2026-08-08 07:06:04 on dist-test-slave-8tdl"
server_key: "d2f3b9f1e4778a7cdca01ec011eef5b2"
server_key_iv: "7fa59b932ebe512630c7ae767418abd1"
server_key_version: "encryptionkey@0"
I20260808 07:06:04.909092 670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "4900b6f337e24d1b9e05e81294fdce0b"
format_stamp: "Formatted at 2026-08-08 07:06:04 on dist-test-slave-8tdl"
server_key: "d2f3b9f1e4778a7cdca01ec011eef5b2"
server_key_iv: "7fa59b932ebe512630c7ae767418abd1"
server_key_version: "encryptionkey@0"
I20260808 07:06:04.911375 670 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260808 07:06:04.912484 686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:04.912806 670 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:04.912954 670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "4900b6f337e24d1b9e05e81294fdce0b"
format_stamp: "Formatted at 2026-08-08 07:06:04 on dist-test-slave-8tdl"
server_key: "d2f3b9f1e4778a7cdca01ec011eef5b2"
server_key_iv: "7fa59b932ebe512630c7ae767418abd1"
server_key_version: "encryptionkey@0"
I20260808 07:06:04.913144 670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:04.958097 670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:04.961040 670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:04.961236 670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:04.961520 670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:04.962102 670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:04.962181 670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:04.962237 670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:04.962283 670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:04.971717 670 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:33679
I20260808 07:06:04.971781 804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:33679 every 8 connection(s)
I20260808 07:06:04.972144 670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:06:04.979127 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 670
I20260808 07:06:04.979322 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:06:04.979592 14140 external_mini_cluster.cc:1428] Setting key f8d993dbce5da056f68a34ea3bc4df98
Aug 08 07:06:04 dist-test-slave-8tdl krb5kdc[555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172764, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:06:04.985157 808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:04.973285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:40919 (local address 127.13.207.1:53209)
0808 07:06:04.973498 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:04.973539 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:04.974050 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:04.974489 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:04.974492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:04.975167 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:04.975391 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:04.975396 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:04.976442 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:04.976443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:04.977399 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:04.977402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:04.977895 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:04.977897 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:04.978040 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:04.978814 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:04.978824 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:04.980684 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:04.982847 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:04.982863 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:04.982866 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:04.983333 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:04.983651 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:04.983652 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:04.983652 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:04.983837 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:04.984381 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:04.984402 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:04.984612 (+ 210us) client_negotiation.cc:769] Sending connection context
0808 07:06:04.984732 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 07:06:04.984899 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":118,"threads_started":1}
I20260808 07:06:04.985157 646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:04.973371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:53209 (local address 127.13.207.62:40919)
0808 07:06:04.973467 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:04.973470 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:04.973661 (+ 191us) server_negotiation.cc:407] Connection header received
0808 07:06:04.974240 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:04.974242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:04.974267 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:04.974383 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:04.975566 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:04.976293 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:04.977535 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:04.977784 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:04.977859 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:04.980787 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:04.980789 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:04.980790 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:04.980804 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:04.982738 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:04.983471 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:04.983472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:04.983472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:04.983567 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:04.984010 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:04.984011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:04.984012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:04.984194 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:04.984280 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:04.984777 (+ 497us) server_negotiation.cc:299] Negotiation successful
0808 07:06:04.984912 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 07:06:04.985900 805 heartbeater.cc:347] Connected to a master server at 127.13.207.62:40919
I20260808 07:06:04.986073 805 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:04.986340 805 heartbeater.cc:511] Master 127.13.207.62:40919 requested a full tablet report, sending...
I20260808 07:06:04.987056 604 ts_manager.cc:194] Registered new tserver with Master: 4900b6f337e24d1b9e05e81294fdce0b (127.13.207.1:33679)
I20260808 07:06:04.987756 604 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:53209
WARNING: no policy specified for kudu/127.13.207.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
I20260808 07:06:05.027273 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40919
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:42999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:05.140560 811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:05.140833 811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:05.140862 811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:05.142480 811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:05.142647 811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:05.142751 811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:06:05.145613 811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:42999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:05.146005 811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:05.146582 811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:05.153376 818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:05.153694 820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:05.153694 817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:05.153731 811 server_base.cc:1034] running on GCE node
I20260808 07:06:05.154405 811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:05.154793 811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:05.155988 811 hybrid_clock.cc:648] HybridClock initialized: now 1786172765155952 us; error 49 us; skew 500 ppm
Aug 08 07:06:05 dist-test-slave-8tdl krb5kdc[555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172765, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:05.161079 811 init.cc:377] Logged in from keytab as kudu/127.13.207.2@KRBTEST.COM (short username kudu)
I20260808 07:06:05.161800 811 webserver.cc:460] Webserver started at http://127.13.207.2:33631/ using document root <none> and password file <none>
I20260808 07:06:05.162078 811 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:05.162129 811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:05.162320 811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:05.163658 811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "0b757f59c2f04667911c1d9808638425"
format_stamp: "Formatted at 2026-08-08 07:06:05 on dist-test-slave-8tdl"
server_key: "3aed0ba21829db2991a59ca3d1b17070"
server_key_iv: "b262a1edeba57c928e11041f5c2330b1"
server_key_version: "encryptionkey@0"
I20260808 07:06:05.164199 811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "0b757f59c2f04667911c1d9808638425"
format_stamp: "Formatted at 2026-08-08 07:06:05 on dist-test-slave-8tdl"
server_key: "3aed0ba21829db2991a59ca3d1b17070"
server_key_iv: "b262a1edeba57c928e11041f5c2330b1"
server_key_version: "encryptionkey@0"
I20260808 07:06:05.166595 811 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260808 07:06:05.167955 827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:05.168357 811 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:05.168481 811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "0b757f59c2f04667911c1d9808638425"
format_stamp: "Formatted at 2026-08-08 07:06:05 on dist-test-slave-8tdl"
server_key: "3aed0ba21829db2991a59ca3d1b17070"
server_key_iv: "b262a1edeba57c928e11041f5c2330b1"
server_key_version: "encryptionkey@0"
I20260808 07:06:05.168596 811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:05.189734 811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:05.193017 811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:05.193205 811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:05.193586 811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:05.193994 811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:05.194217 811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:05.194506 811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:05.194542 811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:05.204805 811 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:45865
I20260808 07:06:05.205050 944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:45865 every 8 connection(s)
I20260808 07:06:05.205428 811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:06:05.205596 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 811
I20260808 07:06:05.205698 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:06:05.205942 14140 external_mini_cluster.cc:1428] Setting key 10c721883203f103bb8fb689fb9b5a5a
Aug 08 07:06:05 dist-test-slave-8tdl krb5kdc[555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172765, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:06:05.219599 947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:05.206488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:40919 (local address 127.13.207.2:48041)
0808 07:06:05.206825 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:05.206875 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:05.207869 (+ 994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:05.208423 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:05.208428 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:05.209487 (+ 1059us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:05.209816 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:05.209823 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.210811 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.210813 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:05.211706 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:05.211710 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.212307 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.212308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:05.212433 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:05.213133 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:05.213144 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:05.214747 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:05.217229 (+ 2482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:05.217243 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:05.217246 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:05.217539 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:05.217917 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:05.217918 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:05.217918 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:05.218061 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:05.218646 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:05.218654 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:05.219053 (+ 399us) client_negotiation.cc:769] Sending connection context
0808 07:06:05.219169 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 07:06:05.219384 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":244,"threads_started":1}
WARNING: no policy specified for kudu/127.13.207.3@KRBTEST.COM; defaulting to no policy
I20260808 07:06:05.219647 646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:05.206479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:48041 (local address 127.13.207.62:40919)
0808 07:06:05.206550 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:05.206552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:05.206975 (+ 423us) server_negotiation.cc:407] Connection header received
0808 07:06:05.208045 (+ 1070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:05.208046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:05.208059 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:05.208252 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:05.210065 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.210631 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.211821 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.212184 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.212339 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:05.214898 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:05.214900 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:05.214901 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:05.214912 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:05.217089 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:05.217629 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:05.217631 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:05.217631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:05.217781 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:05.218182 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:05.218183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:05.218184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:05.218533 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:05.218636 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:05.219215 (+ 579us) server_negotiation.cc:299] Negotiation successful
0808 07:06:05.219351 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:06:05.220527 945 heartbeater.cc:347] Connected to a master server at 127.13.207.62:40919
I20260808 07:06:05.220650 945 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:05.220912 945 heartbeater.cc:511] Master 127.13.207.62:40919 requested a full tablet report, sending...
I20260808 07:06:05.221657 604 ts_manager.cc:194] Registered new tserver with Master: 0b757f59c2f04667911c1d9808638425 (127.13.207.2:45865)
I20260808 07:06:05.222203 604 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:48041
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
I20260808 07:06:05.247933 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40919
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:42999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:05.345561 950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:05.345788 950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:05.345816 950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:05.347182 950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:05.347239 950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:05.347271 950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:06:05.348928 950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:42999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:05.349201 950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:05.349475 950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:05.355182 955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:05.355175 958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:05.355175 956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:05.355418 950 server_base.cc:1034] running on GCE node
I20260808 07:06:05.355746 950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:05.355980 950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:05.357164 950 hybrid_clock.cc:648] HybridClock initialized: now 1786172765357150 us; error 30 us; skew 500 ppm
Aug 08 07:06:05 dist-test-slave-8tdl krb5kdc[555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172765, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:05.360540 950 init.cc:377] Logged in from keytab as kudu/127.13.207.3@KRBTEST.COM (short username kudu)
I20260808 07:06:05.361114 950 webserver.cc:460] Webserver started at http://127.13.207.3:45889/ using document root <none> and password file <none>
I20260808 07:06:05.361359 950 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:05.361429 950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:05.361595 950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:05.362874 950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "633ff123d5914edc898b7089eefc337c"
format_stamp: "Formatted at 2026-08-08 07:06:05 on dist-test-slave-8tdl"
server_key: "af16552d732ff2d7b52e57a46b047a3c"
server_key_iv: "6c61ce58d204d80b570bf0c3a7ce5bf7"
server_key_version: "encryptionkey@0"
I20260808 07:06:05.363248 950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "633ff123d5914edc898b7089eefc337c"
format_stamp: "Formatted at 2026-08-08 07:06:05 on dist-test-slave-8tdl"
server_key: "af16552d732ff2d7b52e57a46b047a3c"
server_key_iv: "6c61ce58d204d80b570bf0c3a7ce5bf7"
server_key_version: "encryptionkey@0"
I20260808 07:06:05.364885 950 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:05.365849 966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:05.366143 950 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:05.366250 950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "633ff123d5914edc898b7089eefc337c"
format_stamp: "Formatted at 2026-08-08 07:06:05 on dist-test-slave-8tdl"
server_key: "af16552d732ff2d7b52e57a46b047a3c"
server_key_iv: "6c61ce58d204d80b570bf0c3a7ce5bf7"
server_key_version: "encryptionkey@0"
I20260808 07:06:05.366318 950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:05.387876 950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:05.390604 950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:05.390777 950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:05.391103 950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:05.391425 950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:05.391464 950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:05.391491 950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:05.391505 950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:05.400152 950 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:35891
I20260808 07:06:05.400195 1083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:35891 every 8 connection(s)
I20260808 07:06:05.400755 950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:06:05.405977 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 950
I20260808 07:06:05.406092 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:06:05.406322 14140 external_mini_cluster.cc:1428] Setting key 853c7f075905d8fd9f047d8e412e5016
Aug 08 07:06:05 dist-test-slave-8tdl krb5kdc[555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172765, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:06:05.412114 1087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:05.401742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:40919 (local address 127.13.207.3:46095)
0808 07:06:05.401932 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:05.401987 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:05.402519 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:05.402819 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:05.402823 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:05.403558 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:05.403783 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:05.403789 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.404538 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.404539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:05.405387 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:05.405389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.405793 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.405793 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:05.405928 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:05.406633 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:05.406644 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:05.408661 (+ 2017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:05.410353 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:05.410363 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:05.410365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:05.410658 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:05.410940 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:05.410942 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:05.410942 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:05.411097 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:05.411490 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:05.411498 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:05.411647 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 07:06:05.411754 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 07:06:05.411917 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":106,"threads_started":1}
I20260808 07:06:05.412292 646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:05.401967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:46095 (local address 127.13.207.62:40919)
0808 07:06:05.402033 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:05.402036 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:05.402075 (+ 39us) server_negotiation.cc:407] Connection header received
0808 07:06:05.402634 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:05.402635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:05.402648 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:05.402723 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:05.403915 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.404431 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.405501 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.405701 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.405778 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:05.408764 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:05.408766 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:05.408766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:05.408775 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:05.410260 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:05.410789 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:05.410790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:05.410790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:05.410853 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:05.411207 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:05.411208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:05.411209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:05.411362 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:05.411439 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:05.412012 (+ 573us) server_negotiation.cc:299] Negotiation successful
0808 07:06:05.412095 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 07:06:05.412858 1084 heartbeater.cc:347] Connected to a master server at 127.13.207.62:40919
I20260808 07:06:05.413028 1084 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:05.413220 1084 heartbeater.cc:511] Master 127.13.207.62:40919 requested a full tablet report, sending...
I20260808 07:06:05.413672 604 ts_manager.cc:194] Registered new tserver with Master: 633ff123d5914edc898b7089eefc337c (127.13.207.3:35891)
I20260808 07:06:05.414330 604 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:46095
I20260808 07:06:05.419627 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:06:05.430282 646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:05.421351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50550 (local address 127.13.207.62:40919)
0808 07:06:05.421488 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:05.421490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:05.421562 (+ 72us) server_negotiation.cc:407] Connection header received
0808 07:06:05.421624 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:05.421624 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:05.421634 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:05.421685 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:05.422515 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.423184 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.424099 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.424304 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.424374 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:05.426838 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:05.426840 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:05.426840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:05.426850 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:05.428559 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:05.429113 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:05.429114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:05.429115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:05.429184 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:05.429531 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:05.429532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:05.429532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:05.429702 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:05.429806 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:05.430052 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 07:06:05.430171 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260808 07:06:05.432766 604 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50550:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:06:05.434798 604 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:05.442977 1100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:05.438556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35296 (local address 127.13.207.2:45865)
0808 07:06:05.438770 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:05.438777 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:05.438790 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:06:05.438909 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:05.438911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:05.438925 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:05.439014 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:05.439259 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.440152 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.441580 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.442387 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.442456 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:05.442525 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:05.442827 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 07:06:05.442877 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":88,"threads_started":1}
I20260808 07:06:05.442979 1098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:05.438312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:45865 (local address 127.0.0.1:35296)
0808 07:06:05.438501 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:05.438512 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:05.438578 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:05.439047 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:05.439050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:05.439052 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:05.439139 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:05.439141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.440256 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.440257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:05.441476 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:05.441479 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.442561 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.442562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:05.442634 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:05.442660 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 07:06:05.442781 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 07:06:05.442852 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":98,"threads_started":1}
I20260808 07:06:05.443231 1101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:05.438630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38682 (local address 127.13.207.1:33679)
0808 07:06:05.439000 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:05.439005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:05.439015 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:06:05.439112 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:05.439115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:05.439130 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:05.439222 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:05.439411 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.440618 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.441832 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.442349 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.442443 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:05.442619 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:05.443042 (+ 423us) server_negotiation.cc:299] Negotiation successful
0808 07:06:05.443102 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":110,"threads_started":1}
I20260808 07:06:05.443264 1099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:05.438527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:33679 (local address 127.0.0.1:38682)
0808 07:06:05.438762 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:05.438774 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:05.438825 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:05.439260 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:05.439261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:05.439264 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:05.439313 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:05.439315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.440732 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.440733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:05.441732 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:05.441735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.442443 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.442444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:05.442653 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:05.442666 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 07:06:05.442808 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 07:06:05.442918 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":4,"thread_start_us":92,"threads_started":1}
I20260808 07:06:05.443464 1096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:05.437905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:35891 (local address 127.0.0.1:37720)
0808 07:06:05.438157 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:05.438211 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:05.438283 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:05.439572 (+ 1289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:05.439573 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:05.439575 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:05.439627 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:05.439628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.441070 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.441070 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:05.442050 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:05.442053 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.443088 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.443093 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:05.443166 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:05.443171 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:06:05.443204 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 07:06:05.443253 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":2,"thread_start_us":139,"threads_started":1}
I20260808 07:06:05.443846 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:05.438096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37720 (local address 127.13.207.3:35891)
0808 07:06:05.439387 (+ 1291us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:05.439391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:05.439402 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:06:05.439441 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:05.439444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:05.439461 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:05.439552 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:05.439698 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.440851 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.442174 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.442953 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.443644 (+ 691us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:05.443715 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:05.443737 (+ 22us) server_negotiation.cc:299] Negotiation successful
0808 07:06:05.443774 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1235,"thread_start_us":114,"threads_started":1}
I20260808 07:06:05.444459 1015 tablet_service.cc:1467] Processing CreateTablet for tablet 71b6ce4c174542d186651083a5290eb3 (DEFAULT_TABLE table=test-table [id=c14cbb407aa54002914fb4dc3626a98e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:05.444451 735 tablet_service.cc:1467] Processing CreateTablet for tablet 71b6ce4c174542d186651083a5290eb3 (DEFAULT_TABLE table=test-table [id=c14cbb407aa54002914fb4dc3626a98e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:05.444824 1015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71b6ce4c174542d186651083a5290eb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:05.444824 735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71b6ce4c174542d186651083a5290eb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:05.447594 873 tablet_service.cc:1467] Processing CreateTablet for tablet 71b6ce4c174542d186651083a5290eb3 (DEFAULT_TABLE table=test-table [id=c14cbb407aa54002914fb4dc3626a98e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:05.448045 873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71b6ce4c174542d186651083a5290eb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:05.448565 1102 tablet_bootstrap.cc:492] T 71b6ce4c174542d186651083a5290eb3 P 633ff123d5914edc898b7089eefc337c: Bootstrap starting.
I20260808 07:06:05.449076 1103 tablet_bootstrap.cc:492] T 71b6ce4c174542d186651083a5290eb3 P 4900b6f337e24d1b9e05e81294fdce0b: Bootstrap starting.
I20260808 07:06:05.449420 1102 tablet_bootstrap.cc:654] T 71b6ce4c174542d186651083a5290eb3 P 633ff123d5914edc898b7089eefc337c: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:05.449951 1102 log.cc:824] T 71b6ce4c174542d186651083a5290eb3 P 633ff123d5914edc898b7089eefc337c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:05.450001 1103 tablet_bootstrap.cc:654] T 71b6ce4c174542d186651083a5290eb3 P 4900b6f337e24d1b9e05e81294fdce0b: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:05.450778 1102 tablet_bootstrap.cc:492] T 71b6ce4c174542d186651083a5290eb3 P 633ff123d5914edc898b7089eefc337c: No bootstrap required, opened a new log
I20260808 07:06:05.450765 1103 log.cc:824] T 71b6ce4c174542d186651083a5290eb3 P 4900b6f337e24d1b9e05e81294fdce0b: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:05.450955 1102 ts_tablet_manager.cc:1397] T 71b6ce4c174542d186651083a5290eb3 P 633ff123d5914edc898b7089eefc337c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:06:05.451601 1103 tablet_bootstrap.cc:492] T 71b6ce4c174542d186651083a5290eb3 P 4900b6f337e24d1b9e05e81294fdce0b: No bootstrap required, opened a new log
I20260808 07:06:05.451674 1103 ts_tablet_manager.cc:1397] T 71b6ce4c174542d186651083a5290eb3 P 4900b6f337e24d1b9e05e81294fdce0b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 07:06:05.452293 1107 tablet_bootstrap.cc:492] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425: Bootstrap starting.
I20260808 07:06:05.452960 1107 tablet_bootstrap.cc:654] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:05.453465 1107 log.cc:824] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:05.453347 1102 raft_consensus.cc:348] T 71b6ce4c174542d186651083a5290eb3 P 633ff123d5914edc898b7089eefc337c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633ff123d5914edc898b7089eefc337c" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 35891 } } peers { permanent_uuid: "0b757f59c2f04667911c1d9808638425" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 45865 } } peers { permanent_uuid: "4900b6f337e24d1b9e05e81294fdce0b" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 33679 } }
I20260808 07:06:05.453557 1102 raft_consensus.cc:374] T 71b6ce4c174542d186651083a5290eb3 P 633ff123d5914edc898b7089eefc337c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:05.453595 1102 raft_consensus.cc:729] T 71b6ce4c174542d186651083a5290eb3 P 633ff123d5914edc898b7089eefc337c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 633ff123d5914edc898b7089eefc337c, State: Initialized, Role: FOLLOWER
I20260808 07:06:05.453725 1102 consensus_queue.cc:260] T 71b6ce4c174542d186651083a5290eb3 P 633ff123d5914edc898b7089eefc337c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633ff123d5914edc898b7089eefc337c" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 35891 } } peers { permanent_uuid: "0b757f59c2f04667911c1d9808638425" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 45865 } } peers { permanent_uuid: "4900b6f337e24d1b9e05e81294fdce0b" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 33679 } }
I20260808 07:06:05.453830 1103 raft_consensus.cc:348] T 71b6ce4c174542d186651083a5290eb3 P 4900b6f337e24d1b9e05e81294fdce0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633ff123d5914edc898b7089eefc337c" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 35891 } } peers { permanent_uuid: "0b757f59c2f04667911c1d9808638425" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 45865 } } peers { permanent_uuid: "4900b6f337e24d1b9e05e81294fdce0b" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 33679 } }
I20260808 07:06:05.453997 1103 raft_consensus.cc:374] T 71b6ce4c174542d186651083a5290eb3 P 4900b6f337e24d1b9e05e81294fdce0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:05.454032 1102 ts_tablet_manager.cc:1428] T 71b6ce4c174542d186651083a5290eb3 P 633ff123d5914edc898b7089eefc337c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:06:05.454032 1103 raft_consensus.cc:729] T 71b6ce4c174542d186651083a5290eb3 P 4900b6f337e24d1b9e05e81294fdce0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4900b6f337e24d1b9e05e81294fdce0b, State: Initialized, Role: FOLLOWER
I20260808 07:06:05.454078 1084 heartbeater.cc:503] Master 127.13.207.62:40919 was elected leader, sending a full tablet report...
I20260808 07:06:05.454159 1103 consensus_queue.cc:260] T 71b6ce4c174542d186651083a5290eb3 P 4900b6f337e24d1b9e05e81294fdce0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633ff123d5914edc898b7089eefc337c" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 35891 } } peers { permanent_uuid: "0b757f59c2f04667911c1d9808638425" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 45865 } } peers { permanent_uuid: "4900b6f337e24d1b9e05e81294fdce0b" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 33679 } }
I20260808 07:06:05.454322 1107 tablet_bootstrap.cc:492] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425: No bootstrap required, opened a new log
I20260808 07:06:05.454391 1107 ts_tablet_manager.cc:1397] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:06:05.454478 1103 ts_tablet_manager.cc:1428] T 71b6ce4c174542d186651083a5290eb3 P 4900b6f337e24d1b9e05e81294fdce0b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:06:05.454509 805 heartbeater.cc:503] Master 127.13.207.62:40919 was elected leader, sending a full tablet report...
I20260808 07:06:05.456692 1107 raft_consensus.cc:348] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633ff123d5914edc898b7089eefc337c" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 35891 } } peers { permanent_uuid: "0b757f59c2f04667911c1d9808638425" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 45865 } } peers { permanent_uuid: "4900b6f337e24d1b9e05e81294fdce0b" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 33679 } }
I20260808 07:06:05.456862 1107 raft_consensus.cc:374] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:05.456888 1107 raft_consensus.cc:729] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b757f59c2f04667911c1d9808638425, State: Initialized, Role: FOLLOWER
I20260808 07:06:05.457013 1107 consensus_queue.cc:260] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633ff123d5914edc898b7089eefc337c" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 35891 } } peers { permanent_uuid: "0b757f59c2f04667911c1d9808638425" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 45865 } } peers { permanent_uuid: "4900b6f337e24d1b9e05e81294fdce0b" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 33679 } }
I20260808 07:06:05.457326 1107 ts_tablet_manager.cc:1428] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:06:05.457373 945 heartbeater.cc:503] Master 127.13.207.62:40919 was elected leader, sending a full tablet report...
W20260808 07:06:05.473238 806 tablet.cc:2369] T 71b6ce4c174542d186651083a5290eb3 P 4900b6f337e24d1b9e05e81294fdce0b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:06:05.651551 1085 tablet.cc:2369] T 71b6ce4c174542d186651083a5290eb3 P 633ff123d5914edc898b7089eefc337c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:06:05.702842 1111 raft_consensus.cc:482] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:05.703032 1111 raft_consensus.cc:504] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633ff123d5914edc898b7089eefc337c" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 35891 } } peers { permanent_uuid: "0b757f59c2f04667911c1d9808638425" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 45865 } } peers { permanent_uuid: "4900b6f337e24d1b9e05e81294fdce0b" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 33679 } }
I20260808 07:06:05.703593 1111 leader_election.cc:288] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 633ff123d5914edc898b7089eefc337c (127.13.207.3:35891), 4900b6f337e24d1b9e05e81294fdce0b (127.13.207.1:33679)
W20260808 07:06:05.706334 946 tablet.cc:2369] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:06:05.707336 947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:05.703926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:35891 (local address 127.13.207.2:46775)
0808 07:06:05.704038 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:05.704054 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:05.704123 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:05.704493 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:05.704494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:05.704496 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:05.704591 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:05.704593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.705322 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.705324 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:05.706187 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:05.706190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.707007 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.707008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:05.707110 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:05.707113 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:05.707186 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 07:06:05.707247 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"mutex_wait_us":1}
I20260808 07:06:05.707333 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:05.704248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:46775 (local address 127.13.207.3:35891)
0808 07:06:05.704342 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:05.704345 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:05.704355 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:06:05.704390 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:05.704392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:05.704404 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:05.704488 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:05.704684 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.705228 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.706321 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.706898 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.706998 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:05.707012 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:05.707197 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 07:06:05.707248 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 07:06:05.707773 1036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71b6ce4c174542d186651083a5290eb3" candidate_uuid: "0b757f59c2f04667911c1d9808638425" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "633ff123d5914edc898b7089eefc337c" is_pre_election: true
I20260808 07:06:05.707921 1112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:05.704116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:33679 (local address 127.13.207.2:38637)
0808 07:06:05.704271 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:05.704281 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:05.704331 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:05.704844 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:05.704846 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:05.704848 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:05.704999 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:05.705001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.705727 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.705728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:05.706762 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:05.706766 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.707628 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.707629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:05.707711 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:05.707721 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 07:06:05.707791 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 07:06:05.707835 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":95,"threads_started":1}
I20260808 07:06:05.707911 1101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:05.704244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:38637 (local address 127.13.207.1:33679)
0808 07:06:05.704319 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:05.704321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:05.704333 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:06:05.704513 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:05.704515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:05.704537 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:05.704835 (+ 298us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:05.705100 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.705635 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.706876 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.707537 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.707626 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:05.707636 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:05.707797 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 07:06:05.707844 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:06:05.707973 1036 raft_consensus.cc:2454] T 71b6ce4c174542d186651083a5290eb3 P 633ff123d5914edc898b7089eefc337c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b757f59c2f04667911c1d9808638425 in term 0.
I20260808 07:06:05.708348 831 leader_election.cc:302] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b757f59c2f04667911c1d9808638425, 633ff123d5914edc898b7089eefc337c; no voters:
I20260808 07:06:05.708524 1111 raft_consensus.cc:2781] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:05.708487 756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71b6ce4c174542d186651083a5290eb3" candidate_uuid: "0b757f59c2f04667911c1d9808638425" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4900b6f337e24d1b9e05e81294fdce0b" is_pre_election: true
I20260808 07:06:05.708595 1111 raft_consensus.cc:482] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:05.708631 1111 raft_consensus.cc:3037] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:05.708670 756 raft_consensus.cc:2454] T 71b6ce4c174542d186651083a5290eb3 P 4900b6f337e24d1b9e05e81294fdce0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b757f59c2f04667911c1d9808638425 in term 0.
I20260808 07:06:05.709676 1111 raft_consensus.cc:504] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633ff123d5914edc898b7089eefc337c" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 35891 } } peers { permanent_uuid: "0b757f59c2f04667911c1d9808638425" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 45865 } } peers { permanent_uuid: "4900b6f337e24d1b9e05e81294fdce0b" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 33679 } }
I20260808 07:06:05.709919 1111 leader_election.cc:288] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425 [CANDIDATE]: Term 1 election: Requested vote from peers 633ff123d5914edc898b7089eefc337c (127.13.207.3:35891), 4900b6f337e24d1b9e05e81294fdce0b (127.13.207.1:33679)
I20260808 07:06:05.710163 1036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71b6ce4c174542d186651083a5290eb3" candidate_uuid: "0b757f59c2f04667911c1d9808638425" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "633ff123d5914edc898b7089eefc337c"
I20260808 07:06:05.710291 1036 raft_consensus.cc:3037] T 71b6ce4c174542d186651083a5290eb3 P 633ff123d5914edc898b7089eefc337c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:05.710320 756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71b6ce4c174542d186651083a5290eb3" candidate_uuid: "0b757f59c2f04667911c1d9808638425" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4900b6f337e24d1b9e05e81294fdce0b"
I20260808 07:06:05.710428 756 raft_consensus.cc:3037] T 71b6ce4c174542d186651083a5290eb3 P 4900b6f337e24d1b9e05e81294fdce0b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:05.711272 1036 raft_consensus.cc:2454] T 71b6ce4c174542d186651083a5290eb3 P 633ff123d5914edc898b7089eefc337c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b757f59c2f04667911c1d9808638425 in term 1.
I20260808 07:06:05.711283 756 raft_consensus.cc:2454] T 71b6ce4c174542d186651083a5290eb3 P 4900b6f337e24d1b9e05e81294fdce0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b757f59c2f04667911c1d9808638425 in term 1.
I20260808 07:06:05.711493 831 leader_election.cc:302] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b757f59c2f04667911c1d9808638425, 4900b6f337e24d1b9e05e81294fdce0b; no voters:
I20260808 07:06:05.711619 1111 raft_consensus.cc:2781] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:05.711824 1111 raft_consensus.cc:686] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425 [term 1 LEADER]: Becoming Leader. State: Replica: 0b757f59c2f04667911c1d9808638425, State: Running, Role: LEADER
I20260808 07:06:05.711944 1111 consensus_queue.cc:237] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633ff123d5914edc898b7089eefc337c" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 35891 } } peers { permanent_uuid: "0b757f59c2f04667911c1d9808638425" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 45865 } } peers { permanent_uuid: "4900b6f337e24d1b9e05e81294fdce0b" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 33679 } }
I20260808 07:06:05.712978 604 catalog_manager.cc:5495] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b757f59c2f04667911c1d9808638425 (127.13.207.2). New cstate: current_term: 1 leader_uuid: "0b757f59c2f04667911c1d9808638425" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633ff123d5914edc898b7089eefc337c" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 35891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b757f59c2f04667911c1d9808638425" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 45865 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4900b6f337e24d1b9e05e81294fdce0b" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 33679 } health_report { overall_health: UNKNOWN } } }
I20260808 07:06:05.732381 1100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:05.728490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35298 (local address 127.13.207.2:45865)
0808 07:06:05.728546 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:05.728548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:05.728560 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:06:05.728617 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:05.728618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:05.728645 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:05.728751 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:05.728943 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.729736 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.730887 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.731249 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.731332 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:05.731736 (+ 404us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:05.731979 (+ 243us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:05.732115 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:05.732287 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 07:06:05.732307 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:06:05.735170 1036 raft_consensus.cc:1261] T 71b6ce4c174542d186651083a5290eb3 P 633ff123d5914edc898b7089eefc337c [term 1 FOLLOWER]: Refusing update from remote peer 0b757f59c2f04667911c1d9808638425: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:05.735170 756 raft_consensus.cc:1261] T 71b6ce4c174542d186651083a5290eb3 P 4900b6f337e24d1b9e05e81294fdce0b [term 1 FOLLOWER]: Refusing update from remote peer 0b757f59c2f04667911c1d9808638425: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:05.735831 1113 consensus_queue.cc:1035] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425 [LEADER]: Connected to new peer: Peer: permanent_uuid: "633ff123d5914edc898b7089eefc337c" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 35891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:05.735978 1111 consensus_queue.cc:1035] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4900b6f337e24d1b9e05e81294fdce0b" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:05.741243 1117 mvcc.cc:205] Tried to move back new op lower bound from 7316163648444989440 to 7316163648356642816. Current Snapshot: MvccSnapshot[applied={T|T < 7316163648444989440}]
I20260808 07:06:05.742408 1116 mvcc.cc:205] Tried to move back new op lower bound from 7316163648444989440 to 7316163648356642816. Current Snapshot: MvccSnapshot[applied={T|T < 7316163648444989440}]
I20260808 07:06:05.746199 604 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:05.746486 604 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:05.748242 604 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 38aecd6d834748099582ec389552ae68: Sending DeleteTablet for 3 replicas of tablet 71b6ce4c174542d186651083a5290eb3
I20260808 07:06:05.749053 873 tablet_service.cc:1514] Processing DeleteTablet for tablet 71b6ce4c174542d186651083a5290eb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:05 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:35296
I20260808 07:06:05.749145 1015 tablet_service.cc:1514] Processing DeleteTablet for tablet 71b6ce4c174542d186651083a5290eb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:05 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:37720
I20260808 07:06:05.749410 735 tablet_service.cc:1514] Processing DeleteTablet for tablet 71b6ce4c174542d186651083a5290eb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:05 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:38682
I20260808 07:06:05.749733 1127 tablet_replica.cc:330] T 71b6ce4c174542d186651083a5290eb3 P 4900b6f337e24d1b9e05e81294fdce0b: stopping tablet replica
I20260808 07:06:05.750094 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 670
I20260808 07:06:05.754554 1126 tablet_replica.cc:330] T 71b6ce4c174542d186651083a5290eb3 P 633ff123d5914edc898b7089eefc337c: stopping tablet replica
I20260808 07:06:05.754794 1126 raft_consensus.cc:2229] T 71b6ce4c174542d186651083a5290eb3 P 633ff123d5914edc898b7089eefc337c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:05.755067 1126 raft_consensus.cc:2258] T 71b6ce4c174542d186651083a5290eb3 P 633ff123d5914edc898b7089eefc337c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:05.755301 1125 tablet_replica.cc:330] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425: stopping tablet replica
I20260808 07:06:05.755442 1125 raft_consensus.cc:2229] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:06:05.755589 1125 raft_consensus.cc:2258] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:05.756142 1126 ts_tablet_manager.cc:1905] T 71b6ce4c174542d186651083a5290eb3 P 633ff123d5914edc898b7089eefc337c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:05.756193 1125 ts_tablet_manager.cc:1905] T 71b6ce4c174542d186651083a5290eb3 P 0b757f59c2f04667911c1d9808638425: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:05.758957 1126 ts_tablet_manager.cc:1918] T 71b6ce4c174542d186651083a5290eb3 P 633ff123d5914edc898b7089eefc337c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:05.759172 1126 log.cc:1196] T 71b6ce4c174542d186651083a5290eb3 P 633ff123d5914edc898b7089eefc337c: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/71b6ce4c174542d186651083a5290eb3
I20260808 07:06:05.759800 1126 ts_tablet_manager.cc:1939] T 71b6ce4c174542d186651083a5290eb3 P 633ff123d5914edc898b7089eefc337c: Deleting consensus metadata
W20260808 07:06:05.760077 591 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.207.1:33679 (error 108)
I20260808 07:06:05.760309 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 811
I20260808 07:06:05.760387 591 catalog_manager.cc:4841] TS 633ff123d5914edc898b7089eefc337c (127.13.207.3:35891): tablet 71b6ce4c174542d186651083a5290eb3 (table test-table [id=c14cbb407aa54002914fb4dc3626a98e]) successfully deleted
I20260808 07:06:05.764340 1099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:05.764075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:33679 (local address 127.0.0.1:38690)
0808 07:06:05.764241 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:05.764298 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.1:33679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260808 07:06:05.764590 591 catalog_manager.cc:4584] TS 4900b6f337e24d1b9e05e81294fdce0b (127.13.207.1:33679): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 71b6ce4c174542d186651083a5290eb3: Network error: Client connection negotiation failed: client connection to 127.13.207.1:33679: connect: Connection refused (error 111)
I20260808 07:06:05.769001 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 950
I20260808 07:06:05.769088 1099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:05.768906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:45865 (local address 127.0.0.1:35302)
0808 07:06:05.769002 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:05.769042 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.2:45865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 07:06:05.778069 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 569
2026-08-08T07:06:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2558 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T07:06:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:05Z Disabled control of system clock
I20260808 07:06:05.817379 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:40377
--webserver_interface=127.13.207.62
--webserver_port=0
--builtin_ntp_servers=127.13.207.20:34777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:40377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:06:05.916186 1132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:05.916489 1132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:05.916517 1132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:05.917992 1132 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:05.918049 1132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:05.918064 1132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:05.918080 1132 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:05.918094 1132 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:05.919973 1132 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:34777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:40377
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:40377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:05.920322 1132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:05.920614 1132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:05.927031 1141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:05.927037 1137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:05.927105 1132 server_base.cc:1034] running on GCE node
W20260808 07:06:05.927026 1138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:05.927721 1132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:05.928076 1132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:05.929271 1132 hybrid_clock.cc:648] HybridClock initialized: now 1786172765929250 us; error 40 us; skew 500 ppm
I20260808 07:06:05.931087 1132 webserver.cc:460] Webserver started at http://127.13.207.62:37593/ using document root <none> and password file <none>
I20260808 07:06:05.931376 1132 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:05.931468 1132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:05.931635 1132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:05.933115 1132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "34fb603640c84f71a31b60dfd84f308d"
format_stamp: "Formatted at 2026-08-08 07:06:05 on dist-test-slave-8tdl"
server_key: "291ec112b5e2e1772be9284e6d8d78c2"
server_key_iv: "b33398ea89787725ff8ff2bd5a37af73"
server_key_version: "encryptionkey@0"
I20260808 07:06:05.933667 1132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "34fb603640c84f71a31b60dfd84f308d"
format_stamp: "Formatted at 2026-08-08 07:06:05 on dist-test-slave-8tdl"
server_key: "291ec112b5e2e1772be9284e6d8d78c2"
server_key_iv: "b33398ea89787725ff8ff2bd5a37af73"
server_key_version: "encryptionkey@0"
I20260808 07:06:05.935431 1132 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 07:06:05.936517 1147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:05.936890 1132 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:05.937047 1132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "34fb603640c84f71a31b60dfd84f308d"
format_stamp: "Formatted at 2026-08-08 07:06:05 on dist-test-slave-8tdl"
server_key: "291ec112b5e2e1772be9284e6d8d78c2"
server_key_iv: "b33398ea89787725ff8ff2bd5a37af73"
server_key_version: "encryptionkey@0"
I20260808 07:06:05.937143 1132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:05.964130 1132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:05.964543 1132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:05.964718 1132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:05.969352 1132 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:40377
I20260808 07:06:05.969395 1202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:40377 every 8 connection(s)
I20260808 07:06:05.969813 1132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:05.970940 1203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:05.973737 1203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d: Bootstrap starting.
I20260808 07:06:05.974715 1203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:05.974787 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 1132
I20260808 07:06:05.974908 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:05.975155 14140 external_mini_cluster.cc:1428] Setting key 0334eb389fc8cb5d01c3026447a752e8
I20260808 07:06:05.975186 1203 log.cc:824] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:05.976104 1203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d: No bootstrap required, opened a new log
I20260808 07:06:05.979261 1203 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34fb603640c84f71a31b60dfd84f308d" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40377 } }
I20260808 07:06:05.979426 1203 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:05.979451 1203 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fb603640c84f71a31b60dfd84f308d, State: Initialized, Role: FOLLOWER
I20260808 07:06:05.979596 1203 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34fb603640c84f71a31b60dfd84f308d" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40377 } }
I20260808 07:06:05.979683 1203 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:05.979746 1203 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:05.979817 1203 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:05.980705 1203 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34fb603640c84f71a31b60dfd84f308d" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40377 } }
I20260808 07:06:05.980834 1203 leader_election.cc:302] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34fb603640c84f71a31b60dfd84f308d; no voters:
I20260808 07:06:05.981045 1203 leader_election.cc:288] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:05.981076 1208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:05.981206 1208 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d [term 1 LEADER]: Becoming Leader. State: Replica: 34fb603640c84f71a31b60dfd84f308d, State: Running, Role: LEADER
I20260808 07:06:05.981307 1208 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34fb603640c84f71a31b60dfd84f308d" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40377 } }
I20260808 07:06:05.981393 1203 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:05.982636 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:05.975790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35648 (local address 127.13.207.62:40377)
0808 07:06:05.976032 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:05.976042 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:05.976078 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:06:05.976706 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:05.976716 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:05.976804 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:05.976978 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:05.977213 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.978830 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.979988 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:05.980681 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:05.980834 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:05.981696 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:05.981700 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:05.981700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:05.982011 (+ 311us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:05.982056 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:05.982072 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:05.982137 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:05.982268 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 07:06:05.982364 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":94,"threads_started":1}
I20260808 07:06:05.982663 1210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34fb603640c84f71a31b60dfd84f308d. Latest consensus state: current_term: 1 leader_uuid: "34fb603640c84f71a31b60dfd84f308d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34fb603640c84f71a31b60dfd84f308d" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40377 } } }
I20260808 07:06:05.982791 1210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:05.983026 1209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34fb603640c84f71a31b60dfd84f308d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34fb603640c84f71a31b60dfd84f308d" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40377 } } }
I20260808 07:06:05.983202 1209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d [sys.catalog]: This master's current role is: LEADER
W20260808 07:06:05.983525 1224 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:06:05.983603 1224 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:06:05.983671 1225 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:05.983865 1225 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:05.985674 1225 catalog_manager.cc:1347] Generated new cluster ID: 974a14f6261f402f952ba526b67becf1
I20260808 07:06:05.985741 1225 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:06.015233 1225 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:06.016042 1225 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:06.021214 1225 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d: Generated new TSK 0
I20260808 07:06:06.021613 1225 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:06.027124 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40377
--builtin_ntp_servers=127.13.207.20:34777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:06:06.140892 1228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:06.141139 1228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:06.141166 1228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:06.142508 1228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:06.142562 1228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:06.142591 1228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:06:06.144515 1228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:34777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:06.144833 1228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:06.145459 1228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:06.154017 1234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:06.154017 1235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:06.154264 1228 server_base.cc:1034] running on GCE node
W20260808 07:06:06.154070 1237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:06.154654 1228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:06.155179 1228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:06.156375 1228 hybrid_clock.cc:648] HybridClock initialized: now 1786172766156340 us; error 56 us; skew 500 ppm
I20260808 07:06:06.158241 1228 webserver.cc:460] Webserver started at http://127.13.207.1:37719/ using document root <none> and password file <none>
I20260808 07:06:06.158573 1228 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:06.158655 1228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:06.158859 1228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:06.160454 1228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46"
format_stamp: "Formatted at 2026-08-08 07:06:06 on dist-test-slave-8tdl"
server_key: "99867b617af2ca1fc7fe80b827073236"
server_key_iv: "e52ae9b02f190661f00d92b9242e4c3c"
server_key_version: "encryptionkey@0"
I20260808 07:06:06.160986 1228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46"
format_stamp: "Formatted at 2026-08-08 07:06:06 on dist-test-slave-8tdl"
server_key: "99867b617af2ca1fc7fe80b827073236"
server_key_iv: "e52ae9b02f190661f00d92b9242e4c3c"
server_key_version: "encryptionkey@0"
I20260808 07:06:06.163290 1228 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260808 07:06:06.164978 1243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:06.165253 1228 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:06.165383 1228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46"
format_stamp: "Formatted at 2026-08-08 07:06:06 on dist-test-slave-8tdl"
server_key: "99867b617af2ca1fc7fe80b827073236"
server_key_iv: "e52ae9b02f190661f00d92b9242e4c3c"
server_key_version: "encryptionkey@0"
I20260808 07:06:06.165464 1228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:06.188383 1228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:06.188956 1228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:06.189160 1228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:06.189455 1228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:06.190111 1228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:06.190191 1228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:06.190244 1228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:06.190279 1228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:06.198974 1228 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:39441
I20260808 07:06:06.199179 1361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:39441 every 8 connection(s)
I20260808 07:06:06.199455 1228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:06:06.205663 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 1228
I20260808 07:06:06.205794 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:06:06.206012 14140 external_mini_cluster.cc:1428] Setting key b3ac514b50d8e035edd4aa920d2d181c
I20260808 07:06:06.207530 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:06.200570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:46201 (local address 127.13.207.62:40377)
0808 07:06:06.200640 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:06.200642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:06.200841 (+ 199us) server_negotiation.cc:407] Connection header received
0808 07:06:06.201499 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:06.201501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:06.201512 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:06.201578 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:06.203219 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.203763 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.205121 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.205446 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.205529 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:06.207241 (+ 1712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:06.207243 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:06.207243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:06.207250 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:06.207268 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:06.207273 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:06.207322 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:06.207403 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 07:06:06.207445 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:06:06.207648 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40377
--builtin_ntp_servers=127.13.207.20:34777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 07:06:06.207789 1364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:06.200502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:40377 (local address 127.13.207.1:46201)
0808 07:06:06.200708 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:06.200751 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:06.201300 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:06.201803 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:06.201808 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:06.201818 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:06.202602 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:06.202610 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.204003 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.204004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:06.204930 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:06.204934 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.205579 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.205580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:06.205782 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:06.207050 (+ 1268us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:06:06.207051 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:06.207100 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:06:06.207134 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:06.207323 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:06.207330 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:06.207333 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:06.207399 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 07:06:06.207532 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":102,"threads_started":1}
I20260808 07:06:06.208693 1362 heartbeater.cc:347] Connected to a master server at 127.13.207.62:40377
I20260808 07:06:06.208889 1362 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:06.209215 1362 heartbeater.cc:511] Master 127.13.207.62:40377 requested a full tablet report, sending...
I20260808 07:06:06.210197 1166 ts_manager.cc:194] Registered new tserver with Master: ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46 (127.13.207.1:39441)
I20260808 07:06:06.211524 1166 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.207.1:46201
W20260808 07:06:06.310012 1365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:06.310261 1365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:06.310289 1365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:06.311833 1365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:06.311921 1365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:06.311956 1365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:06:06.314393 1365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:34777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:06.314734 1365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:06.315106 1365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:06:06.320861 1365 server_base.cc:1034] running on GCE node
W20260808 07:06:06.320811 1372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:06.320816 1374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:06.320811 1370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:06.321429 1365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:06.321780 1365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:06.322945 1365 hybrid_clock.cc:648] HybridClock initialized: now 1786172766322925 us; error 39 us; skew 500 ppm
I20260808 07:06:06.324533 1365 webserver.cc:460] Webserver started at http://127.13.207.2:39607/ using document root <none> and password file <none>
I20260808 07:06:06.324860 1365 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:06.324936 1365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:06.325100 1365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:06.326581 1365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "5cc3ee77aa7b4fd982a8d274a584d674"
format_stamp: "Formatted at 2026-08-08 07:06:06 on dist-test-slave-8tdl"
server_key: "5c1827ae09ce9226e4259c96b4aa598a"
server_key_iv: "752ca29ca043f852046f08f818eb7f7e"
server_key_version: "encryptionkey@0"
I20260808 07:06:06.327152 1365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "5cc3ee77aa7b4fd982a8d274a584d674"
format_stamp: "Formatted at 2026-08-08 07:06:06 on dist-test-slave-8tdl"
server_key: "5c1827ae09ce9226e4259c96b4aa598a"
server_key_iv: "752ca29ca043f852046f08f818eb7f7e"
server_key_version: "encryptionkey@0"
I20260808 07:06:06.328882 1365 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 07:06:06.330317 1381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:06.330852 1365 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:06.330979 1365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "5cc3ee77aa7b4fd982a8d274a584d674"
format_stamp: "Formatted at 2026-08-08 07:06:06 on dist-test-slave-8tdl"
server_key: "5c1827ae09ce9226e4259c96b4aa598a"
server_key_iv: "752ca29ca043f852046f08f818eb7f7e"
server_key_version: "encryptionkey@0"
I20260808 07:06:06.331076 1365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:06.346275 1365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:06.346760 1365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:06.346966 1365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:06.347280 1365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:06.347694 1365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:06.347754 1365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:06.347800 1365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:06.347826 1365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:06.355700 1365 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:39025
I20260808 07:06:06.355767 1501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:39025 every 8 connection(s)
I20260808 07:06:06.356155 1365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:06:06.356606 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 1365
I20260808 07:06:06.356710 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:06:06.356942 14140 external_mini_cluster.cc:1428] Setting key 76320d8423e4b80cce0fb6bc9e8073a0
I20260808 07:06:06.358199 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40377
--builtin_ntp_servers=127.13.207.20:34777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 07:06:06.363930 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:06.357383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:59539 (local address 127.13.207.62:40377)
0808 07:06:06.357434 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:06.357437 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:06.357723 (+ 286us) server_negotiation.cc:407] Connection header received
0808 07:06:06.358597 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:06.358598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:06.358612 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:06.358643 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:06.359806 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.360389 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.361838 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.362052 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.362171 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:06.363374 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:06.363376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:06.363376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:06.363384 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:06.363403 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:06.363407 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:06.363529 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:06.363667 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 07:06:06.363711 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:06:06.364135 1504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:06.357335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:40377 (local address 127.13.207.2:59539)
0808 07:06:06.357570 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:06.357617 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:06.358459 (+ 842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:06.358874 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:06.358879 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:06.358892 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:06.359667 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:06.359674 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.360498 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.360499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:06.361714 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:06.361718 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.362187 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.362188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:06.362358 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:06.363159 (+ 801us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:06:06.363159 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:06.363195 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:06:06.363224 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:06.363551 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:06.363560 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:06.363565 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:06:06.363662 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 07:06:06.363810 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":107,"threads_started":1}
I20260808 07:06:06.364832 1502 heartbeater.cc:347] Connected to a master server at 127.13.207.62:40377
I20260808 07:06:06.364960 1502 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:06.365187 1502 heartbeater.cc:511] Master 127.13.207.62:40377 requested a full tablet report, sending...
I20260808 07:06:06.365640 1166 ts_manager.cc:194] Registered new tserver with Master: 5cc3ee77aa7b4fd982a8d274a584d674 (127.13.207.2:39025)
I20260808 07:06:06.366070 1166 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.207.2:59539
W20260808 07:06:06.463801 1505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:06.464105 1505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:06.464154 1505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:06.465644 1505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:06.465747 1505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:06.465790 1505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:06:06.467417 1505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:34777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:40377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:06.467672 1505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:06.467978 1505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:06.474846 1512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:06.475005 1505 server_base.cc:1034] running on GCE node
W20260808 07:06:06.474846 1514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:06.475250 1511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:06.475564 1505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:06.475888 1505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:06.477084 1505 hybrid_clock.cc:648] HybridClock initialized: now 1786172766477068 us; error 31 us; skew 500 ppm
I20260808 07:06:06.478830 1505 webserver.cc:460] Webserver started at http://127.13.207.3:41107/ using document root <none> and password file <none>
I20260808 07:06:06.479135 1505 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:06.479212 1505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:06.479377 1505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:06.480654 1505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "a842144b43c548ff98becccc71c10ccb"
format_stamp: "Formatted at 2026-08-08 07:06:06 on dist-test-slave-8tdl"
server_key: "9f405435c1659009edc6934056cc3992"
server_key_iv: "c4816c58aa23283b2ca14d9a5e13bab5"
server_key_version: "encryptionkey@0"
I20260808 07:06:06.481083 1505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "a842144b43c548ff98becccc71c10ccb"
format_stamp: "Formatted at 2026-08-08 07:06:06 on dist-test-slave-8tdl"
server_key: "9f405435c1659009edc6934056cc3992"
server_key_iv: "c4816c58aa23283b2ca14d9a5e13bab5"
server_key_version: "encryptionkey@0"
I20260808 07:06:06.483193 1505 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 07:06:06.484202 1521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:06.484483 1505 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:06.484614 1505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "a842144b43c548ff98becccc71c10ccb"
format_stamp: "Formatted at 2026-08-08 07:06:06 on dist-test-slave-8tdl"
server_key: "9f405435c1659009edc6934056cc3992"
server_key_iv: "c4816c58aa23283b2ca14d9a5e13bab5"
server_key_version: "encryptionkey@0"
I20260808 07:06:06.484714 1505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:06.506670 1505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:06.507144 1505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:06.507320 1505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:06.507612 1505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:06.507989 1505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:06.508049 1505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:06.508092 1505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:06.508119 1505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:06.516185 1505 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:43143
I20260808 07:06:06.516340 1643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:43143 every 8 connection(s)
I20260808 07:06:06.516855 1505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:06:06.524618 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:06.518341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:36989 (local address 127.13.207.62:40377)
0808 07:06:06.518455 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:06.518457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:06.518468 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:06:06.519139 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:06.519140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:06.519152 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:06.519226 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:06.520008 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.520583 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.521898 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.522242 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.522327 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:06.524043 (+ 1716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:06.524045 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:06.524045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:06.524052 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:06.524070 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:06.524074 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:06.524141 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:06.524509 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 07:06:06.524542 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 07:06:06.524952 1647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:06.518085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:40377 (local address 127.13.207.3:36989)
0808 07:06:06.518337 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:06.518383 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:06.519009 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:06.519333 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:06.519337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:06.519347 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:06.519887 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:06.519891 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.520748 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.520750 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:06.521768 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:06.521773 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.522448 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.522451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:06.522792 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:06.523793 (+ 1001us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:06:06.523794 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:06.523844 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:06:06.523883 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:06.524230 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:06.524246 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:06.524251 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:06:06.524478 (+ 227us) client_negotiation.cc:240] Negotiation successful
0808 07:06:06.524612 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":85,"threads_started":1}
I20260808 07:06:06.525836 1644 heartbeater.cc:347] Connected to a master server at 127.13.207.62:40377
I20260808 07:06:06.526077 1644 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:06.526333 1644 heartbeater.cc:511] Master 127.13.207.62:40377 requested a full tablet report, sending...
I20260808 07:06:06.526991 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 1505
I20260808 07:06:06.527020 1166 ts_manager.cc:194] Registered new tserver with Master: a842144b43c548ff98becccc71c10ccb (127.13.207.3:43143)
I20260808 07:06:06.527091 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:06:06.527319 14140 external_mini_cluster.cc:1428] Setting key b56a7e1feb4fba23c7ecb96a7ce613b8
I20260808 07:06:06.527563 1166 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.207.3:36989
I20260808 07:06:06.529215 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:06:06.535019 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:06.531596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35664 (local address 127.13.207.62:40377)
0808 07:06:06.531689 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:06.531691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:06.531794 (+ 103us) server_negotiation.cc:407] Connection header received
0808 07:06:06.531896 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:06.531897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:06.531909 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:06.531974 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:06.532167 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.532692 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.533466 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.533674 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.533752 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:06.534632 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:06.534634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:06.534634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:06.534645 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:06.534660 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:06.534664 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:06.534827 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:06.534912 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 07:06:06.534924 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 07:06:06.537249 1165 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:06:06.539007 1165 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:06.548378 1658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:06.543373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:43143 (local address 127.0.0.1:45790)
0808 07:06:06.543804 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:06.543813 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:06.543862 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:06.544186 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:06.544186 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:06.544188 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:06.544228 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:06.544229 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.545671 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.545673 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:06.546513 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:06.546516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.547814 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.547815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:06.548081 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:06.548104 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 07:06:06.548184 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:06:06.548252 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":3,"thread_start_us":82,"threads_started":1}
I20260808 07:06:06.548382 1660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:06.543484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45790 (local address 127.13.207.3:43143)
0808 07:06:06.543750 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:06.543754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:06.543884 (+ 130us) server_negotiation.cc:407] Connection header received
0808 07:06:06.543944 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:06.543946 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:06.543962 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:06.544064 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:06.544303 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.545570 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.546633 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.547582 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.547666 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:06.547773 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:06.548212 (+ 439us) server_negotiation.cc:299] Negotiation successful
0808 07:06:06.548276 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":143,"threads_started":1}
I20260808 07:06:06.548810 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:06.543165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:39441 (local address 127.0.0.1:33084)
0808 07:06:06.543678 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:06.543688 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:06.543732 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:06.544068 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:06.544068 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:06.544070 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:06.544105 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:06.544106 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.545634 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.545635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:06.547007 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:06.547011 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.548234 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.548235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:06.548286 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:06.548289 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:06.548337 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 07:06:06.548378 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":74,"threads_started":1}
I20260808 07:06:06.548923 1657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:06.542964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33178 (local address 127.13.207.2:39025)
0808 07:06:06.543288 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:06.543294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:06.543509 (+ 215us) server_negotiation.cc:407] Connection header received
0808 07:06:06.543564 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:06.543567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:06.543585 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:06.543664 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:06.543965 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.544907 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.546664 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.547741 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.547808 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:06.547961 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:06.548778 (+ 817us) server_negotiation.cc:299] Negotiation successful
0808 07:06:06.548834 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":169,"threads_started":1}
I20260808 07:06:06.549007 1656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:06.542798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:39025 (local address 127.0.0.1:33178)
0808 07:06:06.543264 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:06.543315 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:06.543395 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:06.543757 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:06.543760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:06.543763 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:06.543882 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:06.543884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.545007 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.545008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:06.546534 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:06.546538 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.548106 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.548112 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:06.548189 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:06.548194 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:06:06.548265 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 07:06:06.548311 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":72,"threads_started":1}
I20260808 07:06:06.549161 1661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:06.543634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33084 (local address 127.13.207.1:39441)
0808 07:06:06.543881 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:06.543885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:06.543897 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:06:06.543942 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:06.543944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:06.543962 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:06.544063 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:06.544168 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.545522 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.547246 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.548034 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.548108 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:06.548197 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:06.548419 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 07:06:06.548470 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":102,"threads_started":1}
I20260808 07:06:06.549475 1574 tablet_service.cc:1467] Processing CreateTablet for tablet 56f28babff8d401db864a53750580d25 (DEFAULT_TABLE table=test-table [id=e3045a7453d441b5928b5e9772b8e539]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:06.549928 1574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56f28babff8d401db864a53750580d25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:06.549953 1433 tablet_service.cc:1467] Processing CreateTablet for tablet 56f28babff8d401db864a53750580d25 (DEFAULT_TABLE table=test-table [id=e3045a7453d441b5928b5e9772b8e539]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:06.550057 1294 tablet_service.cc:1467] Processing CreateTablet for tablet 56f28babff8d401db864a53750580d25 (DEFAULT_TABLE table=test-table [id=e3045a7453d441b5928b5e9772b8e539]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:06.550210 1433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56f28babff8d401db864a53750580d25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:06.550382 1294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56f28babff8d401db864a53750580d25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:06.553900 1662 tablet_bootstrap.cc:492] T 56f28babff8d401db864a53750580d25 P 5cc3ee77aa7b4fd982a8d274a584d674: Bootstrap starting.
I20260808 07:06:06.554921 1665 tablet_bootstrap.cc:492] T 56f28babff8d401db864a53750580d25 P ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46: Bootstrap starting.
I20260808 07:06:06.555310 1662 tablet_bootstrap.cc:654] T 56f28babff8d401db864a53750580d25 P 5cc3ee77aa7b4fd982a8d274a584d674: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:06.555645 1665 tablet_bootstrap.cc:654] T 56f28babff8d401db864a53750580d25 P ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:06.555787 1662 log.cc:824] T 56f28babff8d401db864a53750580d25 P 5cc3ee77aa7b4fd982a8d274a584d674: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:06.555968 1665 log.cc:824] T 56f28babff8d401db864a53750580d25 P ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:06.556428 1663 tablet_bootstrap.cc:492] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb: Bootstrap starting.
I20260808 07:06:06.556862 1662 tablet_bootstrap.cc:492] T 56f28babff8d401db864a53750580d25 P 5cc3ee77aa7b4fd982a8d274a584d674: No bootstrap required, opened a new log
I20260808 07:06:06.556963 1662 ts_tablet_manager.cc:1397] T 56f28babff8d401db864a53750580d25 P 5cc3ee77aa7b4fd982a8d274a584d674: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:06:06.557081 1663 tablet_bootstrap.cc:654] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:06.557389 1663 log.cc:824] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:06.557901 1665 tablet_bootstrap.cc:492] T 56f28babff8d401db864a53750580d25 P ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46: No bootstrap required, opened a new log
I20260808 07:06:06.557996 1665 ts_tablet_manager.cc:1397] T 56f28babff8d401db864a53750580d25 P ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 07:06:06.558398 1663 tablet_bootstrap.cc:492] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb: No bootstrap required, opened a new log
I20260808 07:06:06.558482 1663 ts_tablet_manager.cc:1397] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:06:06.560123 1662 raft_consensus.cc:348] T 56f28babff8d401db864a53750580d25 P 5cc3ee77aa7b4fd982a8d274a584d674 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc3ee77aa7b4fd982a8d274a584d674" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39025 } } peers { permanent_uuid: "ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 39441 } } peers { permanent_uuid: "a842144b43c548ff98becccc71c10ccb" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43143 } }
I20260808 07:06:06.560315 1662 raft_consensus.cc:374] T 56f28babff8d401db864a53750580d25 P 5cc3ee77aa7b4fd982a8d274a584d674 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:06.560364 1662 raft_consensus.cc:729] T 56f28babff8d401db864a53750580d25 P 5cc3ee77aa7b4fd982a8d274a584d674 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc3ee77aa7b4fd982a8d274a584d674, State: Initialized, Role: FOLLOWER
I20260808 07:06:06.560511 1662 consensus_queue.cc:260] T 56f28babff8d401db864a53750580d25 P 5cc3ee77aa7b4fd982a8d274a584d674 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc3ee77aa7b4fd982a8d274a584d674" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39025 } } peers { permanent_uuid: "ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 39441 } } peers { permanent_uuid: "a842144b43c548ff98becccc71c10ccb" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43143 } }
I20260808 07:06:06.560849 1662 ts_tablet_manager.cc:1428] T 56f28babff8d401db864a53750580d25 P 5cc3ee77aa7b4fd982a8d274a584d674: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 07:06:06.560976 1502 heartbeater.cc:503] Master 127.13.207.62:40377 was elected leader, sending a full tablet report...
I20260808 07:06:06.561757 1665 raft_consensus.cc:348] T 56f28babff8d401db864a53750580d25 P ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc3ee77aa7b4fd982a8d274a584d674" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39025 } } peers { permanent_uuid: "ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 39441 } } peers { permanent_uuid: "a842144b43c548ff98becccc71c10ccb" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43143 } }
I20260808 07:06:06.561815 1663 raft_consensus.cc:348] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc3ee77aa7b4fd982a8d274a584d674" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39025 } } peers { permanent_uuid: "ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 39441 } } peers { permanent_uuid: "a842144b43c548ff98becccc71c10ccb" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43143 } }
I20260808 07:06:06.561980 1665 raft_consensus.cc:374] T 56f28babff8d401db864a53750580d25 P ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:06.561995 1663 raft_consensus.cc:374] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:06.562038 1663 raft_consensus.cc:729] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a842144b43c548ff98becccc71c10ccb, State: Initialized, Role: FOLLOWER
I20260808 07:06:06.562038 1665 raft_consensus.cc:729] T 56f28babff8d401db864a53750580d25 P ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46, State: Initialized, Role: FOLLOWER
I20260808 07:06:06.562209 1665 consensus_queue.cc:260] T 56f28babff8d401db864a53750580d25 P ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc3ee77aa7b4fd982a8d274a584d674" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39025 } } peers { permanent_uuid: "ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 39441 } } peers { permanent_uuid: "a842144b43c548ff98becccc71c10ccb" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43143 } }
I20260808 07:06:06.562258 1663 consensus_queue.cc:260] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc3ee77aa7b4fd982a8d274a584d674" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39025 } } peers { permanent_uuid: "ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 39441 } } peers { permanent_uuid: "a842144b43c548ff98becccc71c10ccb" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43143 } }
I20260808 07:06:06.562930 1663 ts_tablet_manager.cc:1428] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb: Time spent starting tablet: real 0.004s user 0.004s sys 0.001s
I20260808 07:06:06.562997 1665 ts_tablet_manager.cc:1428] T 56f28babff8d401db864a53750580d25 P ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260808 07:06:06.563062 1362 heartbeater.cc:503] Master 127.13.207.62:40377 was elected leader, sending a full tablet report...
I20260808 07:06:06.563174 1672 raft_consensus.cc:482] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:06.563443 1644 heartbeater.cc:503] Master 127.13.207.62:40377 was elected leader, sending a full tablet report...
I20260808 07:06:06.563279 1672 raft_consensus.cc:504] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc3ee77aa7b4fd982a8d274a584d674" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39025 } } peers { permanent_uuid: "ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 39441 } } peers { permanent_uuid: "a842144b43c548ff98becccc71c10ccb" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43143 } }
I20260808 07:06:06.564774 1672 leader_election.cc:288] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc3ee77aa7b4fd982a8d274a584d674 (127.13.207.2:39025), ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46 (127.13.207.1:39441)
I20260808 07:06:06.568444 1647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:06.564249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:39025 (local address 127.13.207.3:53019)
0808 07:06:06.564298 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:06.564311 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:06.564355 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:06.564527 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:06.564528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:06.564530 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:06.564612 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:06.564614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.565764 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.565765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:06.566912 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:06.566915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.568131 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.568133 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:06.568217 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:06.568221 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:06.568296 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 07:06:06.568347 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 07:06:06.568770 1657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:06.564374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:53019 (local address 127.13.207.2:39025)
0808 07:06:06.564422 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:06.564424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:06.564431 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:06:06.564457 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:06.564458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:06.564469 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:06.564523 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:06.564685 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.565694 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.567049 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.567805 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.567921 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:06.567934 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:06.568427 (+ 493us) server_negotiation.cc:299] Negotiation successful
0808 07:06:06.568477 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:06:06.569200 1455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56f28babff8d401db864a53750580d25" candidate_uuid: "a842144b43c548ff98becccc71c10ccb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc3ee77aa7b4fd982a8d274a584d674" is_pre_election: true
I20260808 07:06:06.569281 1661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:06.564714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:32781 (local address 127.13.207.1:39441)
0808 07:06:06.564753 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:06.564755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:06.565055 (+ 300us) server_negotiation.cc:407] Connection header received
0808 07:06:06.565328 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:06.565330 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:06.565338 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:06.565513 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:06.565721 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.566435 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.568136 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.568901 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.568993 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:06.569003 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:06.569172 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 07:06:06.569223 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:06:06.569285 1675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:06.564521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:39441 (local address 127.13.207.3:32781)
0808 07:06:06.564927 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:06.564938 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:06.565001 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:06.565519 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:06.565520 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:06.565523 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:06.565633 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:06.565635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.566542 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.566544 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:06.567892 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:06.567896 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.569008 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.569009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:06.569082 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:06.569086 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:06.569147 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 07:06:06.569195 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":106,"threads_started":1}
I20260808 07:06:06.569474 1455 raft_consensus.cc:2454] T 56f28babff8d401db864a53750580d25 P 5cc3ee77aa7b4fd982a8d274a584d674 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a842144b43c548ff98becccc71c10ccb in term 0.
I20260808 07:06:06.569710 1315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56f28babff8d401db864a53750580d25" candidate_uuid: "a842144b43c548ff98becccc71c10ccb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46" is_pre_election: true
I20260808 07:06:06.569770 1522 leader_election.cc:302] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc3ee77aa7b4fd982a8d274a584d674, a842144b43c548ff98becccc71c10ccb; no voters:
I20260808 07:06:06.569877 1315 raft_consensus.cc:2454] T 56f28babff8d401db864a53750580d25 P ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a842144b43c548ff98becccc71c10ccb in term 0.
I20260808 07:06:06.570014 1672 raft_consensus.cc:2781] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:06.570101 1672 raft_consensus.cc:482] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:06.570132 1672 raft_consensus.cc:3037] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:06.570989 1672 raft_consensus.cc:504] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc3ee77aa7b4fd982a8d274a584d674" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39025 } } peers { permanent_uuid: "ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 39441 } } peers { permanent_uuid: "a842144b43c548ff98becccc71c10ccb" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43143 } }
I20260808 07:06:06.571144 1672 leader_election.cc:288] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb [CANDIDATE]: Term 1 election: Requested vote from peers 5cc3ee77aa7b4fd982a8d274a584d674 (127.13.207.2:39025), ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46 (127.13.207.1:39441)
I20260808 07:06:06.571350 1315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56f28babff8d401db864a53750580d25" candidate_uuid: "a842144b43c548ff98becccc71c10ccb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46"
I20260808 07:06:06.571380 1455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56f28babff8d401db864a53750580d25" candidate_uuid: "a842144b43c548ff98becccc71c10ccb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc3ee77aa7b4fd982a8d274a584d674"
I20260808 07:06:06.571456 1315 raft_consensus.cc:3037] T 56f28babff8d401db864a53750580d25 P ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:06.571461 1455 raft_consensus.cc:3037] T 56f28babff8d401db864a53750580d25 P 5cc3ee77aa7b4fd982a8d274a584d674 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:06.572489 1315 raft_consensus.cc:2454] T 56f28babff8d401db864a53750580d25 P ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a842144b43c548ff98becccc71c10ccb in term 1.
I20260808 07:06:06.572675 1522 leader_election.cc:302] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a842144b43c548ff98becccc71c10ccb, ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46; no voters:
I20260808 07:06:06.572714 1455 raft_consensus.cc:2454] T 56f28babff8d401db864a53750580d25 P 5cc3ee77aa7b4fd982a8d274a584d674 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a842144b43c548ff98becccc71c10ccb in term 1.
I20260808 07:06:06.572790 1672 raft_consensus.cc:2781] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:06.572862 1672 raft_consensus.cc:686] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb [term 1 LEADER]: Becoming Leader. State: Replica: a842144b43c548ff98becccc71c10ccb, State: Running, Role: LEADER
I20260808 07:06:06.572963 1672 consensus_queue.cc:237] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc3ee77aa7b4fd982a8d274a584d674" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39025 } } peers { permanent_uuid: "ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 39441 } } peers { permanent_uuid: "a842144b43c548ff98becccc71c10ccb" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43143 } }
I20260808 07:06:06.574141 1165 catalog_manager.cc:5495] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb reported cstate change: term changed from 0 to 1, leader changed from <none> to a842144b43c548ff98becccc71c10ccb (127.13.207.3). New cstate: current_term: 1 leader_uuid: "a842144b43c548ff98becccc71c10ccb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc3ee77aa7b4fd982a8d274a584d674" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 39441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a842144b43c548ff98becccc71c10ccb" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43143 } health_report { overall_health: HEALTHY } } }
I20260808 07:06:06.588786 1660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:06.584288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45798 (local address 127.13.207.3:43143)
0808 07:06:06.584416 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:06.584418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:06.584441 (+ 23us) server_negotiation.cc:407] Connection header received
0808 07:06:06.584508 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:06.584509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:06.584552 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:06.584666 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:06.584833 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.585571 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.587025 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:06.587351 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:06.587435 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:06.587899 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:06.588062 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:06.588314 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:06.588620 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 07:06:06.588635 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 07:06:06.591595 1455 raft_consensus.cc:1261] T 56f28babff8d401db864a53750580d25 P 5cc3ee77aa7b4fd982a8d274a584d674 [term 1 FOLLOWER]: Refusing update from remote peer a842144b43c548ff98becccc71c10ccb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:06.591622 1315 raft_consensus.cc:1261] T 56f28babff8d401db864a53750580d25 P ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46 [term 1 FOLLOWER]: Refusing update from remote peer a842144b43c548ff98becccc71c10ccb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:06.592079 1672 consensus_queue.cc:1035] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc3ee77aa7b4fd982a8d274a584d674" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:06.592309 1674 consensus_queue.cc:1035] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 39441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:06.596611 1682 mvcc.cc:205] Tried to move back new op lower bound from 7316163651954237440 to 7316163651883859968. Current Snapshot: MvccSnapshot[applied={T|T < 7316163651954237440}]
I20260808 07:06:06.597247 1680 mvcc.cc:205] Tried to move back new op lower bound from 7316163651954237440 to 7316163651883859968. Current Snapshot: MvccSnapshot[applied={T|T < 7316163651954237440}]
I20260808 07:06:06.598973 1166 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:06.599093 1166 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:06.600374 1166 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34fb603640c84f71a31b60dfd84f308d: Sending DeleteTablet for 3 replicas of tablet 56f28babff8d401db864a53750580d25
I20260808 07:06:06.600894 1574 tablet_service.cc:1514] Processing DeleteTablet for tablet 56f28babff8d401db864a53750580d25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:06 UTC) from {username='slave'} at 127.0.0.1:45790
I20260808 07:06:06.601004 1433 tablet_service.cc:1514] Processing DeleteTablet for tablet 56f28babff8d401db864a53750580d25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:06 UTC) from {username='slave'} at 127.0.0.1:33178
I20260808 07:06:06.601302 1294 tablet_service.cc:1514] Processing DeleteTablet for tablet 56f28babff8d401db864a53750580d25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:06 UTC) from {username='slave'} at 127.0.0.1:33084
I20260808 07:06:06.601511 1690 tablet_replica.cc:330] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb: stopping tablet replica
I20260808 07:06:06.601651 1690 raft_consensus.cc:2229] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:06:06.601783 1690 raft_consensus.cc:2258] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:06.601783 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 1228
I20260808 07:06:06.604421 1690 ts_tablet_manager.cc:1905] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:06.607201 1690 ts_tablet_manager.cc:1918] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:06.607316 1690 log.cc:1196] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/56f28babff8d401db864a53750580d25
I20260808 07:06:06.607596 1690 ts_tablet_manager.cc:1939] T 56f28babff8d401db864a53750580d25 P a842144b43c548ff98becccc71c10ccb: Deleting consensus metadata
I20260808 07:06:06.608350 1148 catalog_manager.cc:4841] TS a842144b43c548ff98becccc71c10ccb (127.13.207.3:43143): tablet 56f28babff8d401db864a53750580d25 (table test-table [id=e3045a7453d441b5928b5e9772b8e539]) successfully deleted
I20260808 07:06:06.610914 1689 tablet_replica.cc:330] T 56f28babff8d401db864a53750580d25 P 5cc3ee77aa7b4fd982a8d274a584d674: stopping tablet replica
I20260808 07:06:06.611111 1689 raft_consensus.cc:2229] T 56f28babff8d401db864a53750580d25 P 5cc3ee77aa7b4fd982a8d274a584d674 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:06.611272 1689 raft_consensus.cc:2258] T 56f28babff8d401db864a53750580d25 P 5cc3ee77aa7b4fd982a8d274a584d674 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:06.612035 1689 ts_tablet_manager.cc:1905] T 56f28babff8d401db864a53750580d25 P 5cc3ee77aa7b4fd982a8d274a584d674: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 07:06:06.612398 1148 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.207.1:39441 (error 108)
W20260808 07:06:06.613137 1656 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:06.612901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:39441 (local address 127.0.0.1:33086)
0808 07:06:06.612961 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:06.612973 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:06.613022 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:06.613072 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.1:39441: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 07:06:06.613402 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 1365
W20260808 07:06:06.613423 1148 catalog_manager.cc:4584] TS ddc9d1a1ffbe4aa4bd8ba4f4f68a8d46 (127.13.207.1:39441): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 56f28babff8d401db864a53750580d25: Network error: Client connection negotiation failed: client connection to 127.13.207.1:39441: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260808 07:06:06.621475 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 1505
I20260808 07:06:06.621681 1656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:06.621470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:39025 (local address 127.0.0.1:33188)
0808 07:06:06.621549 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:06.621629 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.2:39025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 07:06:06.627779 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 1132
2026-08-08T07:06:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (846 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:06 dist-test-slave-8tdl krb5kdc[1696](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:06 dist-test-slave-8tdl krb5kdc[1696](info): set up 2 sockets
Aug 08 07:06:06 dist-test-slave-8tdl krb5kdc[1696](info): commencing operation
krb5kdc: starting...
W20260808 07:06:07.813352 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:06:07 dist-test-slave-8tdl krb5kdc[1696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:06:07.972069 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:35177
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:45745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:35177
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:08.075747 1710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:08.076004 1710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:08.076031 1710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:08.077373 1710 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:08.077441 1710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:08.077456 1710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:08.077469 1710 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:08.077481 1710 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:08.079281 1710 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:45745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:35177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:35177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:08.079574 1710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:08.079845 1710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:08.085409 1715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:08.085405 1718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:08.085405 1716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:08.085873 1710 server_base.cc:1034] running on GCE node
I20260808 07:06:08.086177 1710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:08.086542 1710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:08.087774 1710 hybrid_clock.cc:648] HybridClock initialized: now 1786172768087767 us; error 89 us; skew 500 ppm
Aug 08 07:06:08 dist-test-slave-8tdl krb5kdc[1696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172768, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:08.091341 1710 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:06:08.091908 1710 webserver.cc:460] Webserver started at http://127.13.207.62:34123/ using document root <none> and password file <none>
I20260808 07:06:08.092159 1710 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:08.092211 1710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:08.092314 1710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:08.093775 1710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "919fa483c5034ec588ab683ad7b404c5"
format_stamp: "Formatted at 2026-08-08 07:06:08 on dist-test-slave-8tdl"
server_key: "33531cfa092e40a87d949e2872aafb09"
server_key_iv: "f6be4231d92100455cedf0444d035fd1"
server_key_version: "encryptionkey@0"
I20260808 07:06:08.094221 1710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "919fa483c5034ec588ab683ad7b404c5"
format_stamp: "Formatted at 2026-08-08 07:06:08 on dist-test-slave-8tdl"
server_key: "33531cfa092e40a87d949e2872aafb09"
server_key_iv: "f6be4231d92100455cedf0444d035fd1"
server_key_version: "encryptionkey@0"
I20260808 07:06:08.095956 1710 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 07:06:08.097478 1727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:08.097786 1710 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260808 07:06:08.097890 1710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "919fa483c5034ec588ab683ad7b404c5"
format_stamp: "Formatted at 2026-08-08 07:06:08 on dist-test-slave-8tdl"
server_key: "33531cfa092e40a87d949e2872aafb09"
server_key_iv: "f6be4231d92100455cedf0444d035fd1"
server_key_version: "encryptionkey@0"
I20260808 07:06:08.098083 1710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:08.107105 1710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:08.110064 1710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:08.110219 1710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:08.115660 1710 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:35177
I20260808 07:06:08.115694 1782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:35177 every 8 connection(s)
I20260808 07:06:08.116269 1710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:08.117399 1783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:08.120277 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 1710
I20260808 07:06:08.120442 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:08.120751 14140 external_mini_cluster.cc:1428] Setting key 197936d023046a8257beb4025880d123
I20260808 07:06:08.121794 1783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5: Bootstrap starting.
I20260808 07:06:08.122526 1783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:08.122936 1783 log.cc:824] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:08.123831 1783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5: No bootstrap required, opened a new log
I20260808 07:06:08.126513 1783 raft_consensus.cc:348] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "919fa483c5034ec588ab683ad7b404c5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35177 } }
I20260808 07:06:08.126816 1783 raft_consensus.cc:374] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:08.126878 1783 raft_consensus.cc:729] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 919fa483c5034ec588ab683ad7b404c5, State: Initialized, Role: FOLLOWER
I20260808 07:06:08.127229 1783 consensus_queue.cc:260] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "919fa483c5034ec588ab683ad7b404c5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35177 } }
I20260808 07:06:08.127348 1783 raft_consensus.cc:388] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:08.127381 1783 raft_consensus.cc:482] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:08.127415 1783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:08.128129 1783 raft_consensus.cc:504] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "919fa483c5034ec588ab683ad7b404c5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35177 } }
I20260808 07:06:08.128232 1783 leader_election.cc:302] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 919fa483c5034ec588ab683ad7b404c5; no voters:
I20260808 07:06:08.128468 1783 leader_election.cc:288] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:08.128505 1788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:08.128665 1788 raft_consensus.cc:686] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5 [term 1 LEADER]: Becoming Leader. State: Replica: 919fa483c5034ec588ab683ad7b404c5, State: Running, Role: LEADER
I20260808 07:06:08.128785 1783 sys_catalog.cc:564] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:08.128772 1788 consensus_queue.cc:237] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "919fa483c5034ec588ab683ad7b404c5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35177 } }
I20260808 07:06:08.129221 1788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 919fa483c5034ec588ab683ad7b404c5. Latest consensus state: current_term: 1 leader_uuid: "919fa483c5034ec588ab683ad7b404c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "919fa483c5034ec588ab683ad7b404c5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35177 } } }
I20260808 07:06:08.129318 1788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5 [sys.catalog]: This master's current role is: LEADER
Aug 08 07:06:08 dist-test-slave-8tdl krb5kdc[1696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:06:08.129654 1792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "919fa483c5034ec588ab683ad7b404c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "919fa483c5034ec588ab683ad7b404c5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 35177 } } }
I20260808 07:06:08.129760 1792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:08.129899 1796 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:08.130920 1796 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:08.133157 1796 catalog_manager.cc:1347] Generated new cluster ID: ea63367d059943d6bc77fc3968f4b092
I20260808 07:06:08.133260 1796 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:08.134711 1786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:08.121495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49498 (local address 127.13.207.62:35177)
0808 07:06:08.121958 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:08.122015 (+ 57us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:08.122047 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:06:08.122556 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:08.122563 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:08.122638 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:08.122877 (+ 239us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:08.123599 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.124589 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.125405 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.125870 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.126120 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:08.130193 (+ 4073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:08.130197 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:08.130197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:08.130210 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:08.132618 (+ 2408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:08.133122 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:08.133125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:08.133126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:08.133248 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:08.133560 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:08.133561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:08.133561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:08.133874 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:08.134011 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:08.134210 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 07:06:08.134356 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":137,"threads_started":1}
I20260808 07:06:08.139657 1796 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:08.140506 1796 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:08.143167 1796 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5: Generated new TSK 0
I20260808 07:06:08.143378 1796 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.207.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
Entry for principal kudu/127.13.207.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab.
I20260808 07:06:08.190060 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35177
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:45745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:08.288153 1811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:08.288422 1811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:08.288463 1811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:08.289728 1811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:08.289804 1811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:08.289855 1811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:06:08.291584 1811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:45745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.1.keytab
--principal=kudu/127.13.207.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:08.291886 1811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:08.292198 1811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:08.297995 1817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:08.297982 1818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:08.297984 1820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:08.298556 1811 server_base.cc:1034] running on GCE node
I20260808 07:06:08.298816 1811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:08.299064 1811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:08.300226 1811 hybrid_clock.cc:648] HybridClock initialized: now 1786172768300208 us; error 30 us; skew 500 ppm
Aug 08 07:06:08 dist-test-slave-8tdl krb5kdc[1696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172768, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:08.303257 1811 init.cc:377] Logged in from keytab as kudu/127.13.207.1@KRBTEST.COM (short username kudu)
I20260808 07:06:08.303714 1811 webserver.cc:460] Webserver started at http://127.13.207.1:32797/ using document root <none> and password file <none>
I20260808 07:06:08.303931 1811 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:08.303975 1811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:08.304069 1811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:08.305205 1811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "607295aee6f342ea893af12f96a54b92"
format_stamp: "Formatted at 2026-08-08 07:06:08 on dist-test-slave-8tdl"
server_key: "6bf1ea2e694e4f1a2eadd620f70b20ad"
server_key_iv: "fec53b97c0275043900319720812a1bb"
server_key_version: "encryptionkey@0"
I20260808 07:06:08.305528 1811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "607295aee6f342ea893af12f96a54b92"
format_stamp: "Formatted at 2026-08-08 07:06:08 on dist-test-slave-8tdl"
server_key: "6bf1ea2e694e4f1a2eadd620f70b20ad"
server_key_iv: "fec53b97c0275043900319720812a1bb"
server_key_version: "encryptionkey@0"
I20260808 07:06:08.306994 1811 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:06:08.307788 1827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:08.308048 1811 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:08.308157 1811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "607295aee6f342ea893af12f96a54b92"
format_stamp: "Formatted at 2026-08-08 07:06:08 on dist-test-slave-8tdl"
server_key: "6bf1ea2e694e4f1a2eadd620f70b20ad"
server_key_iv: "fec53b97c0275043900319720812a1bb"
server_key_version: "encryptionkey@0"
I20260808 07:06:08.308220 1811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:08.316823 1811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:08.319162 1811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:08.319291 1811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:08.319502 1811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:08.319839 1811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:08.319877 1811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:08.319902 1811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:08.319916 1811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:08.326356 1811 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:45119
I20260808 07:06:08.326458 1947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:45119 every 8 connection(s)
I20260808 07:06:08.326841 1811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:06:08.327893 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 1811
I20260808 07:06:08.328061 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:06:08.328305 14140 external_mini_cluster.cc:1428] Setting key 41dbc004436465300487fc0add210a87
Aug 08 07:06:08 dist-test-slave-8tdl krb5kdc[1696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172768, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.1@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.207.2@KRBTEST.COM; defaulting to no policy
I20260808 07:06:08.340096 1786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:08.327801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:54923 (local address 127.13.207.62:35177)
0808 07:06:08.327876 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:08.327878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:08.328123 (+ 245us) server_negotiation.cc:407] Connection header received
0808 07:06:08.328870 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:08.328871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:08.328881 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:08.328952 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:08.330437 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.331119 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.332425 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.332767 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.332950 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:08.335816 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:08.335820 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:08.335820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:08.335832 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:08.337763 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:08.338454 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:08.338456 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:08.338456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:08.338531 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:08.339242 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:08.339243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:08.339244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:08.339394 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:08.339469 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:08.339912 (+ 443us) server_negotiation.cc:299] Negotiation successful
0808 07:06:08.340018 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:06:08.340234 1950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:08.327756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35177 (local address 127.13.207.1:54923)
0808 07:06:08.327955 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:08.328018 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:08.328759 (+ 741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:08.329036 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:08.329041 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:08.330011 (+ 970us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:08.330317 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:08.330324 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.331241 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.331242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:08.332250 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:08.332253 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.332970 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.332972 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:08.333115 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:08.333799 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:08.333809 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:08.335687 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:08.337880 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:08.337893 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:08.337896 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:08.338354 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:08.338817 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:08.338820 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:08.338820 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:08.339155 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:08.339512 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:08.339524 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:08.339691 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 07:06:08.339803 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 07:06:08.339985 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":105,"threads_started":1}
I20260808 07:06:08.340785 1948 heartbeater.cc:347] Connected to a master server at 127.13.207.62:35177
I20260808 07:06:08.340906 1948 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:08.341104 1948 heartbeater.cc:511] Master 127.13.207.62:35177 requested a full tablet report, sending...
I20260808 07:06:08.341766 1745 ts_manager.cc:194] Registered new tserver with Master: 607295aee6f342ea893af12f96a54b92 (127.13.207.1:45119)
I20260808 07:06:08.342458 1745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.1@KRBTEST.COM'} at 127.13.207.1:54923
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
Entry for principal kudu/127.13.207.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab.
I20260808 07:06:08.362641 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35177
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:45745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:08.457713 1953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:08.458038 1953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:08.458088 1953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:08.459574 1953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:08.459657 1953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:08.459707 1953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:06:08.461732 1953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:45745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.2.keytab
--principal=kudu/127.13.207.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:08.462109 1953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:08.462402 1953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:08.468511 1960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:08.468441 1963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:08.468462 1958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:08.468866 1953 server_base.cc:1034] running on GCE node
I20260808 07:06:08.469193 1953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:08.469507 1953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:08.470749 1953 hybrid_clock.cc:648] HybridClock initialized: now 1786172768470694 us; error 93 us; skew 500 ppm
Aug 08 07:06:08 dist-test-slave-8tdl krb5kdc[1696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172768, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:08.473922 1953 init.cc:377] Logged in from keytab as kudu/127.13.207.2@KRBTEST.COM (short username kudu)
I20260808 07:06:08.474542 1953 webserver.cc:460] Webserver started at http://127.13.207.2:39643/ using document root <none> and password file <none>
I20260808 07:06:08.474851 1953 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:08.474923 1953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:08.475076 1953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:08.476197 1953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "0a3788935e7449939ac200af25f8cabb"
format_stamp: "Formatted at 2026-08-08 07:06:08 on dist-test-slave-8tdl"
server_key: "0df177e0030c9bc5e320c51755d746d2"
server_key_iv: "f4d9b7044d1b7bd19753c5b63b41d187"
server_key_version: "encryptionkey@0"
I20260808 07:06:08.476598 1953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "0a3788935e7449939ac200af25f8cabb"
format_stamp: "Formatted at 2026-08-08 07:06:08 on dist-test-slave-8tdl"
server_key: "0df177e0030c9bc5e320c51755d746d2"
server_key_iv: "f4d9b7044d1b7bd19753c5b63b41d187"
server_key_version: "encryptionkey@0"
I20260808 07:06:08.478603 1953 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 07:06:08.480633 1971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:08.480939 1953 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:06:08.481070 1953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "0a3788935e7449939ac200af25f8cabb"
format_stamp: "Formatted at 2026-08-08 07:06:08 on dist-test-slave-8tdl"
server_key: "0df177e0030c9bc5e320c51755d746d2"
server_key_iv: "f4d9b7044d1b7bd19753c5b63b41d187"
server_key_version: "encryptionkey@0"
I20260808 07:06:08.481194 1953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:08.506075 1953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:08.508801 1953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:08.509073 1953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:08.509357 1953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:08.509769 1953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:08.509825 1953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:08.509867 1953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:08.509895 1953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:08.517586 1953 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:34221
I20260808 07:06:08.517661 2088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:34221 every 8 connection(s)
I20260808 07:06:08.518028 1953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:06:08.519938 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 1953
I20260808 07:06:08.520224 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:06:08.520457 14140 external_mini_cluster.cc:1428] Setting key 27db5dca2926b1efc90aef3d7ffd6cf8
Aug 08 07:06:08 dist-test-slave-8tdl krb5kdc[1696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172768, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.2@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:06:08.530735 1786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:08.518886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:44495 (local address 127.13.207.62:35177)
0808 07:06:08.518970 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:08.518972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:08.519305 (+ 333us) server_negotiation.cc:407] Connection header received
0808 07:06:08.519928 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:08.519930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:08.519943 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:08.519999 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:08.521466 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.522075 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.523517 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.523724 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.523794 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:08.527159 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:08.527161 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:08.527161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:08.527171 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:08.528992 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:08.529596 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:08.529597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:08.529598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:08.529668 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:08.529966 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:08.529968 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:08.529968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:08.530101 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:08.530179 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:08.530510 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 07:06:08.530605 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 07:06:08.530946 2091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:08.518882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35177 (local address 127.13.207.2:44495)
0808 07:06:08.519176 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:08.519218 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:08.519805 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:08.520076 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:08.520079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:08.521053 (+ 974us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:08.521366 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:08.521372 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.522186 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.522187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:08.523401 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:08.523406 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.523827 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.523828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:08.524004 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:08.524666 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:08.524678 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:08.527019 (+ 2341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:08.529110 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:08.529122 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:08.529124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:08.529501 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:08.529738 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:08.529739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:08.529740 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:08.529882 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:08.530169 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:08.530177 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:08.530343 (+ 166us) client_negotiation.cc:769] Sending connection context
0808 07:06:08.530458 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 07:06:08.530647 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for kudu/127.13.207.3@KRBTEST.COM; defaulting to no policy
I20260808 07:06:08.531581 2089 heartbeater.cc:347] Connected to a master server at 127.13.207.62:35177
I20260808 07:06:08.531701 2089 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:08.531900 2089 heartbeater.cc:511] Master 127.13.207.62:35177 requested a full tablet report, sending...
I20260808 07:06:08.532502 1745 ts_manager.cc:194] Registered new tserver with Master: 0a3788935e7449939ac200af25f8cabb (127.13.207.2:34221)
I20260808 07:06:08.532984 1745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.2@KRBTEST.COM'} at 127.13.207.2:44495
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
Entry for principal kudu/127.13.207.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab.
I20260808 07:06:08.558797 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35177
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:45745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:08.658043 2094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:08.658330 2094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:08.658380 2094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:08.660188 2094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:08.660315 2094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:08.660379 2094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:06:08.662830 2094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:45745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.3.keytab
--principal=kudu/127.13.207.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:35177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:08.663196 2094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:08.663529 2094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:08.669934 2100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:08.669945 2101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:08.670073 2104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:08.671337 2094 server_base.cc:1034] running on GCE node
I20260808 07:06:08.671731 2094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:08.672066 2094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:08.673262 2094 hybrid_clock.cc:648] HybridClock initialized: now 1786172768673221 us; error 49 us; skew 500 ppm
Aug 08 07:06:08 dist-test-slave-8tdl krb5kdc[1696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172768, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:08.676775 2094 init.cc:377] Logged in from keytab as kudu/127.13.207.3@KRBTEST.COM (short username kudu)
I20260808 07:06:08.677415 2094 webserver.cc:460] Webserver started at http://127.13.207.3:36357/ using document root <none> and password file <none>
I20260808 07:06:08.677691 2094 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:08.677744 2094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:08.677899 2094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:08.679217 2094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "9d43bf5c24614304a5d993ec6015fcf9"
format_stamp: "Formatted at 2026-08-08 07:06:08 on dist-test-slave-8tdl"
server_key: "c7f0c14ced32b59f0787aaa039b308b2"
server_key_iv: "5126d8458220b1ac184be12422f1be2f"
server_key_version: "encryptionkey@0"
I20260808 07:06:08.679647 2094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "9d43bf5c24614304a5d993ec6015fcf9"
format_stamp: "Formatted at 2026-08-08 07:06:08 on dist-test-slave-8tdl"
server_key: "c7f0c14ced32b59f0787aaa039b308b2"
server_key_iv: "5126d8458220b1ac184be12422f1be2f"
server_key_version: "encryptionkey@0"
I20260808 07:06:08.681277 2094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:06:08.682382 2111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:08.682854 2094 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:08.682967 2094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "9d43bf5c24614304a5d993ec6015fcf9"
format_stamp: "Formatted at 2026-08-08 07:06:08 on dist-test-slave-8tdl"
server_key: "c7f0c14ced32b59f0787aaa039b308b2"
server_key_iv: "5126d8458220b1ac184be12422f1be2f"
server_key_version: "encryptionkey@0"
I20260808 07:06:08.683073 2094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:08.697582 2094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:08.700245 2094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:08.700430 2094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:08.700708 2094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:08.701061 2094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:08.701116 2094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:08.701172 2094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:08.701224 2094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:08.707784 2094 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:33461
I20260808 07:06:08.707827 2229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:33461 every 8 connection(s)
I20260808 07:06:08.708199 2094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:06:08 dist-test-slave-8tdl krb5kdc[1696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172768, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.3@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:06:08.716084 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 2094
I20260808 07:06:08.716174 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:06:08.716434 14140 external_mini_cluster.cc:1428] Setting key eddaeb66c7189fb52dad808a13992298
I20260808 07:06:08.719826 1786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:08.708996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:56979 (local address 127.13.207.62:35177)
0808 07:06:08.709075 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:08.709078 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:08.709331 (+ 253us) server_negotiation.cc:407] Connection header received
0808 07:06:08.709914 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:08.709916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:08.709929 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:08.709981 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:08.711248 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.711899 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.713054 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.713283 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.713363 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:08.716060 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:08.716062 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:08.716062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:08.716069 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:08.718268 (+ 2199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:08.718785 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:08.718785 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:08.718786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:08.718853 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:08.719125 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:08.719125 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:08.719126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:08.719257 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:08.719327 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:08.719634 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 07:06:08.719725 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:06:08.720058 2232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:08.709011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:35177 (local address 127.13.207.3:56979)
0808 07:06:08.709204 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:08.709244 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:08.709794 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:08.710065 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:08.710069 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:08.710867 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:08.711110 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:08.711116 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.712018 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.712019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:08.712934 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:08.712937 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.713400 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.713401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:08.713541 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:08.714168 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:08.714177 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:08.715930 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:08.718364 (+ 2434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:08.718380 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:08.718382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:08.718650 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:08.718921 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:08.718922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:08.718923 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:08.719050 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:08.719333 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:08.719341 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:08.719491 (+ 150us) client_negotiation.cc:769] Sending connection context
0808 07:06:08.719600 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 07:06:08.719793 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":91,"threads_started":1}
I20260808 07:06:08.720651 2230 heartbeater.cc:347] Connected to a master server at 127.13.207.62:35177
I20260808 07:06:08.720750 2230 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:08.720964 2230 heartbeater.cc:511] Master 127.13.207.62:35177 requested a full tablet report, sending...
I20260808 07:06:08.721310 1745 ts_manager.cc:194] Registered new tserver with Master: 9d43bf5c24614304a5d993ec6015fcf9 (127.13.207.3:33461)
I20260808 07:06:08.721737 1745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.207.3@KRBTEST.COM'} at 127.13.207.3:56979
I20260808 07:06:08.729485 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:06:08.739005 1786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:08.731048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49514 (local address 127.13.207.62:35177)
0808 07:06:08.731124 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:08.731126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:08.731236 (+ 110us) server_negotiation.cc:407] Connection header received
0808 07:06:08.731286 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:08.731287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:08.731297 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:08.731394 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:08.731954 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.732718 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.733593 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.733809 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.733886 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:08.735429 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:08.735430 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:08.735431 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:08.735440 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:08.737161 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:08.737653 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:08.737655 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:08.737655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:08.737724 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:08.738019 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:08.738082 (+ 63us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:08.738082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:08.738397 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:08.738492 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:08.738766 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 07:06:08.738863 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 07:06:08.741427 1745 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:06:08.743427 1745 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:08.751928 2246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:08.747282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54256 (local address 127.13.207.3:33461)
0808 07:06:08.747554 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:08.747559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:08.747616 (+ 57us) server_negotiation.cc:407] Connection header received
0808 07:06:08.747783 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:08.747786 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:08.747803 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:08.747886 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:08.748276 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.749395 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.750762 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.751412 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.751537 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:08.751670 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:08.751770 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 07:06:08.751822 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":114,"threads_started":1}
I20260808 07:06:08.751935 2243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:08.747132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:33461 (local address 127.0.0.1:54256)
0808 07:06:08.747565 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:08.747575 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:08.747624 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:08.748040 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:08.748042 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:08.748045 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:08.748143 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:08.748145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.749534 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.749535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:08.750632 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:08.750634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.751542 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.751543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:08.751646 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:08.751664 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 07:06:08.751749 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 07:06:08.751821 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"mutex_wait_us":34,"thread_start_us":85,"threads_started":1}
I20260808 07:06:08.752382 2245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:08.746901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:34221 (local address 127.0.0.1:37584)
0808 07:06:08.747286 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:08.747300 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:08.747468 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:08.748121 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:08.748122 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:08.748124 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:08.748182 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:08.748192 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.749495 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.749497 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:08.750697 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:08.750712 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.752021 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.752022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:08.752086 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:08.752090 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:08.752213 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 07:06:08.752273 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":52,"thread_start_us":90,"threads_started":1}
I20260808 07:06:08.752729 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:08.746543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:45119 (local address 127.0.0.1:59352)
0808 07:06:08.746721 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:08.746765 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:08.746841 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:08.747296 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:08.747300 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:08.747303 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:08.747400 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:08.747403 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.748822 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.748824 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:08.750314 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:08.750317 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.751773 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.751774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:08.751847 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:08.751852 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:06:08.752596 (+ 744us) client_negotiation.cc:240] Negotiation successful
0808 07:06:08.752655 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":100,"threads_started":1}
I20260808 07:06:08.753010 2242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:08.746724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59352 (local address 127.13.207.1:45119)
0808 07:06:08.746976 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:08.746982 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:08.746994 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:06:08.747037 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:08.747039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:08.747055 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:08.747177 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:08.747505 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.748691 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.750421 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.751580 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.751668 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:08.751730 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:08.752871 (+ 1141us) server_negotiation.cc:299] Negotiation successful
0808 07:06:08.752926 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":108,"threads_started":1}
I20260808 07:06:08.752925 2160 tablet_service.cc:1467] Processing CreateTablet for tablet d665530915da461dbd229964cfe900e1 (DEFAULT_TABLE table=test-table [id=6aa27c9a3e3a42cd81bb1ce4c6cbf645]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:08.753335 2244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:08.747061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37584 (local address 127.13.207.2:34221)
0808 07:06:08.747760 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:08.747764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:08.747775 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:06:08.747813 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:08.747815 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:08.747832 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:08.747900 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:08.748352 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.749361 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.750818 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.751854 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.751924 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:08.751984 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:08.753188 (+ 1204us) server_negotiation.cc:299] Negotiation successful
0808 07:06:08.753250 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":148,"threads_started":1}
I20260808 07:06:08.753607 2160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d665530915da461dbd229964cfe900e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:08.753832 1878 tablet_service.cc:1467] Processing CreateTablet for tablet d665530915da461dbd229964cfe900e1 (DEFAULT_TABLE table=test-table [id=6aa27c9a3e3a42cd81bb1ce4c6cbf645]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:08.754133 2019 tablet_service.cc:1467] Processing CreateTablet for tablet d665530915da461dbd229964cfe900e1 (DEFAULT_TABLE table=test-table [id=6aa27c9a3e3a42cd81bb1ce4c6cbf645]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:08.754420 1878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d665530915da461dbd229964cfe900e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:08.754518 2019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d665530915da461dbd229964cfe900e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:08.757417 2247 tablet_bootstrap.cc:492] T d665530915da461dbd229964cfe900e1 P 607295aee6f342ea893af12f96a54b92: Bootstrap starting.
I20260808 07:06:08.758255 2247 tablet_bootstrap.cc:654] T d665530915da461dbd229964cfe900e1 P 607295aee6f342ea893af12f96a54b92: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:08.758745 2249 tablet_bootstrap.cc:492] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb: Bootstrap starting.
I20260808 07:06:08.758868 2248 tablet_bootstrap.cc:492] T d665530915da461dbd229964cfe900e1 P 9d43bf5c24614304a5d993ec6015fcf9: Bootstrap starting.
I20260808 07:06:08.758931 2247 log.cc:824] T d665530915da461dbd229964cfe900e1 P 607295aee6f342ea893af12f96a54b92: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:08.759559 2248 tablet_bootstrap.cc:654] T d665530915da461dbd229964cfe900e1 P 9d43bf5c24614304a5d993ec6015fcf9: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:08.759675 2249 tablet_bootstrap.cc:654] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:08.759955 2247 tablet_bootstrap.cc:492] T d665530915da461dbd229964cfe900e1 P 607295aee6f342ea893af12f96a54b92: No bootstrap required, opened a new log
I20260808 07:06:08.760046 2247 ts_tablet_manager.cc:1397] T d665530915da461dbd229964cfe900e1 P 607295aee6f342ea893af12f96a54b92: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 07:06:08.760185 2248 log.cc:824] T d665530915da461dbd229964cfe900e1 P 9d43bf5c24614304a5d993ec6015fcf9: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:08.760375 2249 log.cc:824] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:08.761188 2248 tablet_bootstrap.cc:492] T d665530915da461dbd229964cfe900e1 P 9d43bf5c24614304a5d993ec6015fcf9: No bootstrap required, opened a new log
I20260808 07:06:08.761273 2248 ts_tablet_manager.cc:1397] T d665530915da461dbd229964cfe900e1 P 9d43bf5c24614304a5d993ec6015fcf9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:06:08.761385 2249 tablet_bootstrap.cc:492] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb: No bootstrap required, opened a new log
I20260808 07:06:08.761505 2249 ts_tablet_manager.cc:1397] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:06:08.762097 2247 raft_consensus.cc:348] T d665530915da461dbd229964cfe900e1 P 607295aee6f342ea893af12f96a54b92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "607295aee6f342ea893af12f96a54b92" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45119 } } peers { permanent_uuid: "0a3788935e7449939ac200af25f8cabb" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34221 } } peers { permanent_uuid: "9d43bf5c24614304a5d993ec6015fcf9" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33461 } }
I20260808 07:06:08.762283 2247 raft_consensus.cc:374] T d665530915da461dbd229964cfe900e1 P 607295aee6f342ea893af12f96a54b92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:08.762331 2247 raft_consensus.cc:729] T d665530915da461dbd229964cfe900e1 P 607295aee6f342ea893af12f96a54b92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 607295aee6f342ea893af12f96a54b92, State: Initialized, Role: FOLLOWER
I20260808 07:06:08.762454 2247 consensus_queue.cc:260] T d665530915da461dbd229964cfe900e1 P 607295aee6f342ea893af12f96a54b92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "607295aee6f342ea893af12f96a54b92" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45119 } } peers { permanent_uuid: "0a3788935e7449939ac200af25f8cabb" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34221 } } peers { permanent_uuid: "9d43bf5c24614304a5d993ec6015fcf9" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33461 } }
I20260808 07:06:08.762816 2247 ts_tablet_manager.cc:1428] T d665530915da461dbd229964cfe900e1 P 607295aee6f342ea893af12f96a54b92: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:06:08.762971 1948 heartbeater.cc:503] Master 127.13.207.62:35177 was elected leader, sending a full tablet report...
I20260808 07:06:08.763458 2248 raft_consensus.cc:348] T d665530915da461dbd229964cfe900e1 P 9d43bf5c24614304a5d993ec6015fcf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "607295aee6f342ea893af12f96a54b92" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45119 } } peers { permanent_uuid: "0a3788935e7449939ac200af25f8cabb" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34221 } } peers { permanent_uuid: "9d43bf5c24614304a5d993ec6015fcf9" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33461 } }
I20260808 07:06:08.763655 2248 raft_consensus.cc:374] T d665530915da461dbd229964cfe900e1 P 9d43bf5c24614304a5d993ec6015fcf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:08.763710 2248 raft_consensus.cc:729] T d665530915da461dbd229964cfe900e1 P 9d43bf5c24614304a5d993ec6015fcf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d43bf5c24614304a5d993ec6015fcf9, State: Initialized, Role: FOLLOWER
I20260808 07:06:08.763659 2249 raft_consensus.cc:348] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "607295aee6f342ea893af12f96a54b92" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45119 } } peers { permanent_uuid: "0a3788935e7449939ac200af25f8cabb" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34221 } } peers { permanent_uuid: "9d43bf5c24614304a5d993ec6015fcf9" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33461 } }
I20260808 07:06:08.763832 2249 raft_consensus.cc:374] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:08.763876 2249 raft_consensus.cc:729] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3788935e7449939ac200af25f8cabb, State: Initialized, Role: FOLLOWER
I20260808 07:06:08.763859 2248 consensus_queue.cc:260] T d665530915da461dbd229964cfe900e1 P 9d43bf5c24614304a5d993ec6015fcf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "607295aee6f342ea893af12f96a54b92" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45119 } } peers { permanent_uuid: "0a3788935e7449939ac200af25f8cabb" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34221 } } peers { permanent_uuid: "9d43bf5c24614304a5d993ec6015fcf9" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33461 } }
I20260808 07:06:08.763984 2249 consensus_queue.cc:260] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "607295aee6f342ea893af12f96a54b92" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45119 } } peers { permanent_uuid: "0a3788935e7449939ac200af25f8cabb" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34221 } } peers { permanent_uuid: "9d43bf5c24614304a5d993ec6015fcf9" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33461 } }
I20260808 07:06:08.764191 2248 ts_tablet_manager.cc:1428] T d665530915da461dbd229964cfe900e1 P 9d43bf5c24614304a5d993ec6015fcf9: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:06:08.764209 2230 heartbeater.cc:503] Master 127.13.207.62:35177 was elected leader, sending a full tablet report...
I20260808 07:06:08.764269 2249 ts_tablet_manager.cc:1428] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:06:08.764403 2089 heartbeater.cc:503] Master 127.13.207.62:35177 was elected leader, sending a full tablet report...
W20260808 07:06:08.768671 2090 tablet.cc:2369] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:06:08.827841 1949 tablet.cc:2369] T d665530915da461dbd229964cfe900e1 P 607295aee6f342ea893af12f96a54b92: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:06:08.900847 2256 raft_consensus.cc:482] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:08.901058 2256 raft_consensus.cc:504] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "607295aee6f342ea893af12f96a54b92" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45119 } } peers { permanent_uuid: "0a3788935e7449939ac200af25f8cabb" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34221 } } peers { permanent_uuid: "9d43bf5c24614304a5d993ec6015fcf9" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33461 } }
I20260808 07:06:08.901485 2256 leader_election.cc:288] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 607295aee6f342ea893af12f96a54b92 (127.13.207.1:45119), 9d43bf5c24614304a5d993ec6015fcf9 (127.13.207.3:33461)
I20260808 07:06:08.905071 2242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:08.901899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:43513 (local address 127.13.207.1:45119)
0808 07:06:08.901953 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:08.901955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:08.901967 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:06:08.902003 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:08.902004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:08.902017 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:08.902093 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:08.902308 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.903132 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.904181 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.904601 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.904689 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:08.904699 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:08.904929 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 07:06:08.904992 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:06:08.905069 2091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:08.901768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:45119 (local address 127.13.207.2:43513)
0808 07:06:08.901827 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:08.901840 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:08.901891 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:08.902122 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:08.902123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:08.902126 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:08.902210 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:08.902212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.903232 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.903233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:08.904078 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:08.904080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.904701 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.904703 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:08.904843 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:08.904847 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:08.904922 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 07:06:08.904975 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 07:06:08.905506 1899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d665530915da461dbd229964cfe900e1" candidate_uuid: "0a3788935e7449939ac200af25f8cabb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "607295aee6f342ea893af12f96a54b92" is_pre_election: true
I20260808 07:06:08.905598 2257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:08.901910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:33461 (local address 127.13.207.2:53463)
0808 07:06:08.902182 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:08.902191 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:08.902241 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:08.902519 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:08.902521 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:08.902524 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:08.902609 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:08.902612 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.903604 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.903606 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:08.904622 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:08.904625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.905339 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.905340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:08.905416 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:08.905420 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:08.905475 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 07:06:08.905519 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":168,"threads_started":1}
I20260808 07:06:08.905604 2246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:08.902087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:53463 (local address 127.13.207.3:33461)
0808 07:06:08.902176 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:08.902179 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:08.902235 (+ 56us) server_negotiation.cc:407] Connection header received
0808 07:06:08.902350 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:08.902352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:08.902362 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:08.902419 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:08.902753 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.903456 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.904723 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.905208 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.905293 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:08.905303 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:08.905506 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 07:06:08.905551 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 07:06:08.905728 1899 raft_consensus.cc:2454] T d665530915da461dbd229964cfe900e1 P 607295aee6f342ea893af12f96a54b92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a3788935e7449939ac200af25f8cabb in term 0.
I20260808 07:06:08.905978 1974 leader_election.cc:302] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3788935e7449939ac200af25f8cabb, 607295aee6f342ea893af12f96a54b92; no voters:
I20260808 07:06:08.906044 2181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d665530915da461dbd229964cfe900e1" candidate_uuid: "0a3788935e7449939ac200af25f8cabb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d43bf5c24614304a5d993ec6015fcf9" is_pre_election: true
I20260808 07:06:08.906226 2181 raft_consensus.cc:2454] T d665530915da461dbd229964cfe900e1 P 9d43bf5c24614304a5d993ec6015fcf9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a3788935e7449939ac200af25f8cabb in term 0.
I20260808 07:06:08.906273 2256 raft_consensus.cc:2781] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:08.906383 2256 raft_consensus.cc:482] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:08.906423 2256 raft_consensus.cc:3037] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:08.907315 2256 raft_consensus.cc:504] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "607295aee6f342ea893af12f96a54b92" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45119 } } peers { permanent_uuid: "0a3788935e7449939ac200af25f8cabb" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34221 } } peers { permanent_uuid: "9d43bf5c24614304a5d993ec6015fcf9" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33461 } }
I20260808 07:06:08.907473 2256 leader_election.cc:288] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb [CANDIDATE]: Term 1 election: Requested vote from peers 607295aee6f342ea893af12f96a54b92 (127.13.207.1:45119), 9d43bf5c24614304a5d993ec6015fcf9 (127.13.207.3:33461)
I20260808 07:06:08.907779 1899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d665530915da461dbd229964cfe900e1" candidate_uuid: "0a3788935e7449939ac200af25f8cabb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "607295aee6f342ea893af12f96a54b92"
I20260808 07:06:08.907891 1899 raft_consensus.cc:3037] T d665530915da461dbd229964cfe900e1 P 607295aee6f342ea893af12f96a54b92 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:08.907843 2181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d665530915da461dbd229964cfe900e1" candidate_uuid: "0a3788935e7449939ac200af25f8cabb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d43bf5c24614304a5d993ec6015fcf9"
I20260808 07:06:08.908054 2181 raft_consensus.cc:3037] T d665530915da461dbd229964cfe900e1 P 9d43bf5c24614304a5d993ec6015fcf9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:08.908821 1899 raft_consensus.cc:2454] T d665530915da461dbd229964cfe900e1 P 607295aee6f342ea893af12f96a54b92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3788935e7449939ac200af25f8cabb in term 1.
I20260808 07:06:08.909027 1974 leader_election.cc:302] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3788935e7449939ac200af25f8cabb, 607295aee6f342ea893af12f96a54b92; no voters:
I20260808 07:06:08.909128 2256 raft_consensus.cc:2781] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:08.909312 2256 raft_consensus.cc:686] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb [term 1 LEADER]: Becoming Leader. State: Replica: 0a3788935e7449939ac200af25f8cabb, State: Running, Role: LEADER
I20260808 07:06:08.909426 2256 consensus_queue.cc:237] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "607295aee6f342ea893af12f96a54b92" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45119 } } peers { permanent_uuid: "0a3788935e7449939ac200af25f8cabb" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34221 } } peers { permanent_uuid: "9d43bf5c24614304a5d993ec6015fcf9" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33461 } }
I20260808 07:06:08.909528 2181 raft_consensus.cc:2454] T d665530915da461dbd229964cfe900e1 P 9d43bf5c24614304a5d993ec6015fcf9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3788935e7449939ac200af25f8cabb in term 1.
I20260808 07:06:08.910776 1745 catalog_manager.cc:5495] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a3788935e7449939ac200af25f8cabb (127.13.207.2). New cstate: current_term: 1 leader_uuid: "0a3788935e7449939ac200af25f8cabb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "607295aee6f342ea893af12f96a54b92" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3788935e7449939ac200af25f8cabb" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 34221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d43bf5c24614304a5d993ec6015fcf9" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33461 } health_report { overall_health: UNKNOWN } } }
I20260808 07:06:08.940744 2244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:08.937282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37588 (local address 127.13.207.2:34221)
0808 07:06:08.937366 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:08.937368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:08.937419 (+ 51us) server_negotiation.cc:407] Connection header received
0808 07:06:08.937464 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:08.937465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:08.937490 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:08.937574 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:08.937826 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.938480 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.939608 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:08.939811 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:08.939884 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:08.940133 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:08.940260 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:08.940348 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:08.940515 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 07:06:08.940574 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 07:06:08.943235 2181 raft_consensus.cc:1261] T d665530915da461dbd229964cfe900e1 P 9d43bf5c24614304a5d993ec6015fcf9 [term 1 FOLLOWER]: Refusing update from remote peer 0a3788935e7449939ac200af25f8cabb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:08.943239 1899 raft_consensus.cc:1261] T d665530915da461dbd229964cfe900e1 P 607295aee6f342ea893af12f96a54b92 [term 1 FOLLOWER]: Refusing update from remote peer 0a3788935e7449939ac200af25f8cabb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:08.943710 2259 consensus_queue.cc:1035] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d43bf5c24614304a5d993ec6015fcf9" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:08.944011 2256 consensus_queue.cc:1035] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb [LEADER]: Connected to new peer: Peer: permanent_uuid: "607295aee6f342ea893af12f96a54b92" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 45119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:08.948654 2263 mvcc.cc:205] Tried to move back new op lower bound from 7316163661587628032 to 7316163661454065664. Current Snapshot: MvccSnapshot[applied={T|T < 7316163661587628032 or (T in {7316163661587628032})}]
I20260808 07:06:08.948724 2264 mvcc.cc:205] Tried to move back new op lower bound from 7316163661587628032 to 7316163661454065664. Current Snapshot: MvccSnapshot[applied={T|T < 7316163661587628032}]
I20260808 07:06:08.951576 1744 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:08.951720 1744 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:08.952701 1744 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 919fa483c5034ec588ab683ad7b404c5: Sending DeleteTablet for 3 replicas of tablet d665530915da461dbd229964cfe900e1
I20260808 07:06:08.953186 1878 tablet_service.cc:1514] Processing DeleteTablet for tablet d665530915da461dbd229964cfe900e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:08 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:59352
I20260808 07:06:08.953233 2019 tablet_service.cc:1514] Processing DeleteTablet for tablet d665530915da461dbd229964cfe900e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:08 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:37584
I20260808 07:06:08.953351 2160 tablet_service.cc:1514] Processing DeleteTablet for tablet d665530915da461dbd229964cfe900e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:08 UTC) from {username='kudu', principal='kudu/127.13.207.62@KRBTEST.COM'} at 127.0.0.1:54256
I20260808 07:06:08.953567 2272 tablet_replica.cc:330] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb: stopping tablet replica
I20260808 07:06:08.953672 2272 raft_consensus.cc:2229] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:06:08.953784 2272 raft_consensus.cc:2258] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:08.953895 2274 tablet_replica.cc:330] T d665530915da461dbd229964cfe900e1 P 9d43bf5c24614304a5d993ec6015fcf9: stopping tablet replica
I20260808 07:06:08.954068 2274 raft_consensus.cc:2229] T d665530915da461dbd229964cfe900e1 P 9d43bf5c24614304a5d993ec6015fcf9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:08.954260 2274 raft_consensus.cc:2258] T d665530915da461dbd229964cfe900e1 P 9d43bf5c24614304a5d993ec6015fcf9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:08.954257 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 1811
I20260808 07:06:08.954483 2272 ts_tablet_manager.cc:1905] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:08.957878 2272 ts_tablet_manager.cc:1918] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:08.958348 2272 log.cc:1196] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-1/wal/wals/d665530915da461dbd229964cfe900e1
I20260808 07:06:08.958737 2272 ts_tablet_manager.cc:1939] T d665530915da461dbd229964cfe900e1 P 0a3788935e7449939ac200af25f8cabb: Deleting consensus metadata
I20260808 07:06:08.959394 1730 catalog_manager.cc:4841] TS 0a3788935e7449939ac200af25f8cabb (127.13.207.2:34221): tablet d665530915da461dbd229964cfe900e1 (table test-table [id=6aa27c9a3e3a42cd81bb1ce4c6cbf645]) successfully deleted
I20260808 07:06:08.963215 2274 ts_tablet_manager.cc:1905] T d665530915da461dbd229964cfe900e1 P 9d43bf5c24614304a5d993ec6015fcf9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:08.965349 2274 ts_tablet_manager.cc:1918] T d665530915da461dbd229964cfe900e1 P 9d43bf5c24614304a5d993ec6015fcf9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:08.965476 2274 log.cc:1196] T d665530915da461dbd229964cfe900e1 P 9d43bf5c24614304a5d993ec6015fcf9: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/d665530915da461dbd229964cfe900e1
I20260808 07:06:08.965969 2274 ts_tablet_manager.cc:1939] T d665530915da461dbd229964cfe900e1 P 9d43bf5c24614304a5d993ec6015fcf9: Deleting consensus metadata
I20260808 07:06:08.967195 1730 catalog_manager.cc:4841] TS 9d43bf5c24614304a5d993ec6015fcf9 (127.13.207.3:33461): tablet d665530915da461dbd229964cfe900e1 (table test-table [id=6aa27c9a3e3a42cd81bb1ce4c6cbf645]) successfully deleted
W20260808 07:06:08.969475 1730 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.207.1:45119 (error 108)
I20260808 07:06:08.969838 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 1953
I20260808 07:06:08.971619 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:08.969993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:45119 (local address 127.0.0.1:59362)
0808 07:06:08.971499 (+ 1506us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:08.971567 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.1:45119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1474}
W20260808 07:06:08.972162 1730 catalog_manager.cc:4584] TS 607295aee6f342ea893af12f96a54b92 (127.13.207.1:45119): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d665530915da461dbd229964cfe900e1: Network error: Client connection negotiation failed: client connection to 127.13.207.1:45119: connect: Connection refused (error 111)
I20260808 07:06:08.977063 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 2094
I20260808 07:06:08.988162 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 1710
2026-08-08T07:06:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2362 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T07:06:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:09Z Disabled control of system clock
I20260808 07:06:09.026319 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:42169
--webserver_interface=127.13.207.62
--webserver_port=0
--builtin_ntp_servers=127.13.207.20:43591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:42169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:06:09.121665 2279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:09.122192 2279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:09.122246 2279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:09.123708 2279 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:09.123775 2279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:09.123790 2279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:09.123803 2279 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:09.123816 2279 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:09.125347 2279 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:43591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:42169
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:42169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:09.125612 2279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:09.125931 2279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:09.131616 2284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:09.131616 2288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:09.131618 2286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:09.132076 2279 server_base.cc:1034] running on GCE node
I20260808 07:06:09.132367 2279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:09.132776 2279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:09.133997 2279 hybrid_clock.cc:648] HybridClock initialized: now 1786172769133975 us; error 34 us; skew 500 ppm
I20260808 07:06:09.135562 2279 webserver.cc:460] Webserver started at http://127.13.207.62:45007/ using document root <none> and password file <none>
I20260808 07:06:09.135845 2279 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:09.135929 2279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:09.136157 2279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:09.137341 2279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "76852935a51a444789a95963588edbf1"
format_stamp: "Formatted at 2026-08-08 07:06:09 on dist-test-slave-8tdl"
server_key: "11781ab48ecd96f567dbafe1e495e649"
server_key_iv: "26abb589070d1b8aad2ddbf58faa09ce"
server_key_version: "encryptionkey@0"
I20260808 07:06:09.137737 2279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "76852935a51a444789a95963588edbf1"
format_stamp: "Formatted at 2026-08-08 07:06:09 on dist-test-slave-8tdl"
server_key: "11781ab48ecd96f567dbafe1e495e649"
server_key_iv: "26abb589070d1b8aad2ddbf58faa09ce"
server_key_version: "encryptionkey@0"
I20260808 07:06:09.139487 2279 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 07:06:09.140404 2295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:09.140774 2279 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:09.140960 2279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "76852935a51a444789a95963588edbf1"
format_stamp: "Formatted at 2026-08-08 07:06:09 on dist-test-slave-8tdl"
server_key: "11781ab48ecd96f567dbafe1e495e649"
server_key_iv: "26abb589070d1b8aad2ddbf58faa09ce"
server_key_version: "encryptionkey@0"
I20260808 07:06:09.141064 2279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:09.153905 2279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:09.154318 2279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:09.154493 2279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:09.159029 2279 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:42169
I20260808 07:06:09.159062 2348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:42169 every 8 connection(s)
I20260808 07:06:09.159454 2279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:09.160542 2349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:09.162986 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 2279
I20260808 07:06:09.163126 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:09.163159 2349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1: Bootstrap starting.
I20260808 07:06:09.163349 14140 external_mini_cluster.cc:1428] Setting key 3b52309ea4e7bcdf4df185cbcebfcc63
I20260808 07:06:09.163985 2349 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:09.164393 2349 log.cc:824] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:09.165431 2349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1: No bootstrap required, opened a new log
I20260808 07:06:09.167892 2349 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76852935a51a444789a95963588edbf1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 42169 } }
I20260808 07:06:09.168080 2349 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:09.168115 2349 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76852935a51a444789a95963588edbf1, State: Initialized, Role: FOLLOWER
I20260808 07:06:09.168244 2349 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76852935a51a444789a95963588edbf1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 42169 } }
I20260808 07:06:09.168320 2349 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:09.168351 2349 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:09.168421 2349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:09.169324 2349 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76852935a51a444789a95963588edbf1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 42169 } }
I20260808 07:06:09.169461 2349 leader_election.cc:302] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76852935a51a444789a95963588edbf1; no voters:
I20260808 07:06:09.169649 2349 leader_election.cc:288] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:09.169744 2355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:09.169919 2355 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1 [term 1 LEADER]: Becoming Leader. State: Replica: 76852935a51a444789a95963588edbf1, State: Running, Role: LEADER
I20260808 07:06:09.169998 2349 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:09.170022 2355 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76852935a51a444789a95963588edbf1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 42169 } }
I20260808 07:06:09.170333 2352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:09.163947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45830 (local address 127.13.207.62:42169)
0808 07:06:09.164267 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:09.164276 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:09.164318 (+ 42us) server_negotiation.cc:407] Connection header received
0808 07:06:09.164858 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:09.164866 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:09.164945 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:09.165122 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:09.165351 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.166791 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.167783 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.168417 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.168541 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:09.169347 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:09.169351 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:09.169352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:09.169596 (+ 244us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:09.169644 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:09.169661 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:09.169732 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:09.169904 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 07:06:09.170125 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":131,"threads_started":1}
I20260808 07:06:09.170382 2355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76852935a51a444789a95963588edbf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76852935a51a444789a95963588edbf1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 42169 } } }
I20260808 07:06:09.170469 2355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:09.170603 2355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76852935a51a444789a95963588edbf1. Latest consensus state: current_term: 1 leader_uuid: "76852935a51a444789a95963588edbf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76852935a51a444789a95963588edbf1" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 42169 } } }
I20260808 07:06:09.170699 2355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1 [sys.catalog]: This master's current role is: LEADER
W20260808 07:06:09.171943 2372 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:06:09.171994 2372 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:06:09.172057 2360 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:09.172322 2360 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:09.174316 2360 catalog_manager.cc:1347] Generated new cluster ID: fef6045592ab430684fa985c71d872f5
I20260808 07:06:09.174443 2360 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:09.185258 2360 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:09.185902 2360 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:09.192909 2360 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1: Generated new TSK 0
I20260808 07:06:09.193405 2360 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:09.197448 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.1:0
--local_ip_for_outbound_sockets=127.13.207.1
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:42169
--builtin_ntp_servers=127.13.207.20:43591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:06:09.288125 2375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:09.288401 2375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:09.288441 2375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:09.289753 2375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:09.289825 2375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:09.289875 2375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.1
I20260808 07:06:09.291478 2375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:43591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.207.1
--webserver_port=0
--tserver_master_addrs=127.13.207.62:42169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.1
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:09.291783 2375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:09.292137 2375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:09.298705 2384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:09.298847 2381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:09.298776 2382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:09.299115 2375 server_base.cc:1034] running on GCE node
I20260808 07:06:09.299363 2375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:09.299666 2375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:09.300861 2375 hybrid_clock.cc:648] HybridClock initialized: now 1786172769300822 us; error 53 us; skew 500 ppm
I20260808 07:06:09.302642 2375 webserver.cc:460] Webserver started at http://127.13.207.1:33239/ using document root <none> and password file <none>
I20260808 07:06:09.303099 2375 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:09.303195 2375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:09.303372 2375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:09.304637 2375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-0/data/instance:
uuid: "141ad6a3f31f4f52b47fce172cf0f54a"
format_stamp: "Formatted at 2026-08-08 07:06:09 on dist-test-slave-8tdl"
server_key: "c24a88655cd47361ba775aaca36ba362"
server_key_iv: "8ed0666fbf2c377f1fe193e379c1d58f"
server_key_version: "encryptionkey@0"
I20260808 07:06:09.305071 2375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance:
uuid: "141ad6a3f31f4f52b47fce172cf0f54a"
format_stamp: "Formatted at 2026-08-08 07:06:09 on dist-test-slave-8tdl"
server_key: "c24a88655cd47361ba775aaca36ba362"
server_key_iv: "8ed0666fbf2c377f1fe193e379c1d58f"
server_key_version: "encryptionkey@0"
I20260808 07:06:09.306707 2375 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 07:06:09.307698 2390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:09.307891 2375 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:09.308004 2375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-0/wal
uuid: "141ad6a3f31f4f52b47fce172cf0f54a"
format_stamp: "Formatted at 2026-08-08 07:06:09 on dist-test-slave-8tdl"
server_key: "c24a88655cd47361ba775aaca36ba362"
server_key_iv: "8ed0666fbf2c377f1fe193e379c1d58f"
server_key_version: "encryptionkey@0"
I20260808 07:06:09.308094 2375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:09.316201 2375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:09.316764 2375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:09.317031 2375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:09.317339 2375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:09.317711 2375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:09.317770 2375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:09.317816 2375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:09.317858 2375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:09.324672 2375 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.1:39783
I20260808 07:06:09.324723 2512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.1:39783 every 8 connection(s)
I20260808 07:06:09.325102 2375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-0/data/info.pb
I20260808 07:06:09.331178 2352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:09.326012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.1:53749 (local address 127.13.207.62:42169)
0808 07:06:09.326074 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:09.326077 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:09.326236 (+ 159us) server_negotiation.cc:407] Connection header received
0808 07:06:09.326940 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:09.326942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:09.326955 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:09.327032 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:09.327903 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.328452 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.329623 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.329818 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.329883 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:09.330913 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:09.330914 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:09.330914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:09.330920 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:09.330935 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:09.330939 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:09.330998 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:09.331081 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 07:06:09.331106 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 07:06:09.331426 2515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:09.325876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:42169 (local address 127.13.207.1:53749)
0808 07:06:09.326097 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:09.326145 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:09.326803 (+ 658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:09.327111 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:09.327114 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:09.327125 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:09.327805 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:09.327811 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.328586 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.328587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:09.329532 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:09.329534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.329914 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.329915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:09.330056 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:09.330768 (+ 712us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:06:09.330769 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:09.330801 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:06:09.330829 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:09.331008 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:09.331013 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:09.331015 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:06:09.331076 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 07:06:09.331180 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":104,"threads_started":1}
I20260808 07:06:09.331995 2513 heartbeater.cc:347] Connected to a master server at 127.13.207.62:42169
I20260808 07:06:09.332103 2513 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:09.332285 2513 heartbeater.cc:511] Master 127.13.207.62:42169 requested a full tablet report, sending...
I20260808 07:06:09.332883 2312 ts_manager.cc:194] Registered new tserver with Master: 141ad6a3f31f4f52b47fce172cf0f54a (127.13.207.1:39783)
I20260808 07:06:09.333535 2312 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.207.1:53749
I20260808 07:06:09.334007 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 2375
I20260808 07:06:09.334134 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-0/wal/instance
I20260808 07:06:09.334329 14140 external_mini_cluster.cc:1428] Setting key e860a24f76fe594b905d708689418948
I20260808 07:06:09.335491 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.2:0
--local_ip_for_outbound_sockets=127.13.207.2
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:42169
--builtin_ntp_servers=127.13.207.20:43591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:06:09.426959 2516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:09.427284 2516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:09.427347 2516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:09.428773 2516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:09.428856 2516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:09.428906 2516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.2
I20260808 07:06:09.430521 2516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:43591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.207.2
--webserver_port=0
--tserver_master_addrs=127.13.207.62:42169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.2
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:09.430877 2516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:09.431247 2516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:09.437287 2521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:09.437326 2524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:09.437327 2516 server_base.cc:1034] running on GCE node
W20260808 07:06:09.437281 2522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:09.437749 2516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:09.438093 2516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:09.439309 2516 hybrid_clock.cc:648] HybridClock initialized: now 1786172769439278 us; error 39 us; skew 500 ppm
I20260808 07:06:09.441015 2516 webserver.cc:460] Webserver started at http://127.13.207.2:38577/ using document root <none> and password file <none>
I20260808 07:06:09.441321 2516 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:09.441402 2516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:09.441574 2516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:09.442970 2516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-1/data/instance:
uuid: "00514b9406f64ff88ecbbdd3fd0f46ea"
format_stamp: "Formatted at 2026-08-08 07:06:09 on dist-test-slave-8tdl"
server_key: "c7ec8a14b7e596b0cf06a2a0aaba0180"
server_key_iv: "e8bb86b0c5c8e2f7d58b4e58b49451bc"
server_key_version: "encryptionkey@0"
I20260808 07:06:09.443442 2516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance:
uuid: "00514b9406f64ff88ecbbdd3fd0f46ea"
format_stamp: "Formatted at 2026-08-08 07:06:09 on dist-test-slave-8tdl"
server_key: "c7ec8a14b7e596b0cf06a2a0aaba0180"
server_key_iv: "e8bb86b0c5c8e2f7d58b4e58b49451bc"
server_key_version: "encryptionkey@0"
I20260808 07:06:09.445363 2516 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 07:06:09.446408 2530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:09.446813 2516 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:09.446928 2516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-1/wal
uuid: "00514b9406f64ff88ecbbdd3fd0f46ea"
format_stamp: "Formatted at 2026-08-08 07:06:09 on dist-test-slave-8tdl"
server_key: "c7ec8a14b7e596b0cf06a2a0aaba0180"
server_key_iv: "e8bb86b0c5c8e2f7d58b4e58b49451bc"
server_key_version: "encryptionkey@0"
I20260808 07:06:09.447024 2516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:09.463518 2516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:09.463919 2516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:09.464089 2516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:09.464355 2516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:09.464712 2516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:09.464764 2516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:09.464807 2516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:09.464843 2516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:09.472447 2516 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.2:39529
I20260808 07:06:09.472653 2651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.2:39529 every 8 connection(s)
I20260808 07:06:09.473150 2516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-1/data/info.pb
I20260808 07:06:09.478998 2352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:09.474017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:36239 (local address 127.13.207.62:42169)
0808 07:06:09.474065 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:09.474067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:09.474332 (+ 265us) server_negotiation.cc:407] Connection header received
0808 07:06:09.474918 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:09.474920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:09.474931 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:09.474985 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:09.475647 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.476156 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.477207 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.477406 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.477497 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:09.478533 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:09.478535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:09.478535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:09.478541 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:09.478557 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:09.478562 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:09.478630 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:09.478848 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 07:06:09.478932 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:06:09.479102 2654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:09.474014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:42169 (local address 127.13.207.2:36239)
0808 07:06:09.474205 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:09.474242 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:09.474813 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:09.475045 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:09.475048 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:09.475057 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:09.475565 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:09.475569 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.476239 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.476240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:09.477094 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:09.477097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.477520 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.477521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:09.477647 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:09.478329 (+ 682us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:06:09.478330 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:09.478360 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:06:09.478392 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:09.478723 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:09.478731 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:09.478734 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:09.478807 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 07:06:09.478920 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":93,"threads_started":1}
I20260808 07:06:09.479660 2652 heartbeater.cc:347] Connected to a master server at 127.13.207.62:42169
I20260808 07:06:09.479764 2652 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:09.479956 2652 heartbeater.cc:511] Master 127.13.207.62:42169 requested a full tablet report, sending...
I20260808 07:06:09.480530 2312 ts_manager.cc:194] Registered new tserver with Master: 00514b9406f64ff88ecbbdd3fd0f46ea (127.13.207.2:39529)
I20260808 07:06:09.481032 2312 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.207.2:36239
I20260808 07:06:09.482831 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 2516
I20260808 07:06:09.482944 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-1/wal/instance
I20260808 07:06:09.483105 14140 external_mini_cluster.cc:1428] Setting key edc6a03e9dcfbc9ae52c888a80902baa
I20260808 07:06:09.484453 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.207.3:0
--local_ip_for_outbound_sockets=127.13.207.3
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:42169
--builtin_ntp_servers=127.13.207.20:43591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:06:09.587965 2655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:09.588248 2655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:09.588294 2655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:09.589593 2655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:09.589668 2655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:09.589715 2655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.207.3
I20260808 07:06:09.591519 2655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:43591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.207.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.207.3
--webserver_port=0
--tserver_master_addrs=127.13.207.62:42169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.207.3
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:09.591830 2655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:09.592180 2655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:09.597662 2660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:09.597657 2663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:09.597654 2661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:09.597749 2655 server_base.cc:1034] running on GCE node
I20260808 07:06:09.598197 2655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:09.598493 2655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:09.599681 2655 hybrid_clock.cc:648] HybridClock initialized: now 1786172769599653 us; error 47 us; skew 500 ppm
I20260808 07:06:09.601265 2655 webserver.cc:460] Webserver started at http://127.13.207.3:36633/ using document root <none> and password file <none>
I20260808 07:06:09.601548 2655 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:09.601608 2655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:09.601763 2655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:09.603044 2655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-2/data/instance:
uuid: "2ce21617b0bd4edda7a629250f4173ed"
format_stamp: "Formatted at 2026-08-08 07:06:09 on dist-test-slave-8tdl"
server_key: "fa7af361846760a0df0f82be90f9130c"
server_key_iv: "3a672f3b581287ea8536113ffaa603a5"
server_key_version: "encryptionkey@0"
I20260808 07:06:09.603410 2655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance:
uuid: "2ce21617b0bd4edda7a629250f4173ed"
format_stamp: "Formatted at 2026-08-08 07:06:09 on dist-test-slave-8tdl"
server_key: "fa7af361846760a0df0f82be90f9130c"
server_key_iv: "3a672f3b581287ea8536113ffaa603a5"
server_key_version: "encryptionkey@0"
I20260808 07:06:09.605151 2655 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 07:06:09.606060 2669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:09.606316 2655 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:09.606431 2655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-2/wal
uuid: "2ce21617b0bd4edda7a629250f4173ed"
format_stamp: "Formatted at 2026-08-08 07:06:09 on dist-test-slave-8tdl"
server_key: "fa7af361846760a0df0f82be90f9130c"
server_key_iv: "3a672f3b581287ea8536113ffaa603a5"
server_key_version: "encryptionkey@0"
I20260808 07:06:09.606534 2655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:09.641897 2655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:09.642354 2655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:09.642534 2655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:09.642841 2655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:09.643240 2655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:09.643301 2655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:09.643352 2655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:09.643381 2655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:09.650270 2655 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.3:43875
I20260808 07:06:09.650326 2792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.3:43875 every 8 connection(s)
I20260808 07:06:09.650725 2655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-2/data/info.pb
I20260808 07:06:09.652566 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 2655
I20260808 07:06:09.652696 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-2/wal/instance
I20260808 07:06:09.652933 14140 external_mini_cluster.cc:1428] Setting key d050d94bae4d4a8af525a894bad33926
I20260808 07:06:09.657881 2352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:09.651509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.3:55699 (local address 127.13.207.62:42169)
0808 07:06:09.651588 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:09.651590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:09.651833 (+ 243us) server_negotiation.cc:407] Connection header received
0808 07:06:09.652344 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:09.652356 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:09.652375 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:09.652480 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:09.653844 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.654634 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.655838 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.656059 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.656151 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:09.657508 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:09.657510 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:09.657511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:09.657517 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:09.657538 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:09.657543 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:09.657606 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:09.657751 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 07:06:09.657788 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 07:06:09.658141 2795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:09.651469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:42169 (local address 127.13.207.3:55699)
0808 07:06:09.651683 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:09.651725 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:09.652228 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:09.652601 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:09.652606 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:09.652617 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:09.653723 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:09.653732 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.654794 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.654795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:09.655703 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:09.655706 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.656167 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.656168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:09.656542 (+ 374us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:09.657287 (+ 745us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:06:09.657288 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:09.657321 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:06:09.657352 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:09.657609 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:09.657616 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:09.657620 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:09.657751 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 07:06:09.657899 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":95,"threads_started":1}
I20260808 07:06:09.658592 2793 heartbeater.cc:347] Connected to a master server at 127.13.207.62:42169
I20260808 07:06:09.658739 2793 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:09.658929 2793 heartbeater.cc:511] Master 127.13.207.62:42169 requested a full tablet report, sending...
I20260808 07:06:09.659322 2312 ts_manager.cc:194] Registered new tserver with Master: 2ce21617b0bd4edda7a629250f4173ed (127.13.207.3:43875)
I20260808 07:06:09.659677 2312 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.207.3:55699
I20260808 07:06:09.665498 14140 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:06:09.670817 2352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:09.667158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45834 (local address 127.13.207.62:42169)
0808 07:06:09.667278 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:09.667281 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:09.667393 (+ 112us) server_negotiation.cc:407] Connection header received
0808 07:06:09.667476 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:09.667476 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:09.667490 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:09.667543 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:09.667743 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.668307 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.669151 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.669378 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.669471 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:09.670504 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:09.670506 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:09.670506 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:09.670512 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:09.670528 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:09.670532 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:09.670600 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:09.670664 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 07:06:09.670716 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 07:06:09.672744 2312 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:06:09.674575 2312 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:09.681666 2803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:09.677012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.2:39529 (local address 127.0.0.1:47114)
0808 07:06:09.677488 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:09.677504 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:09.677591 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:09.678122 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:09.678123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:09.678126 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:09.678174 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:09.678175 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.679124 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.679125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:09.680423 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:09.680426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.681394 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.681395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:09.681483 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:09.681493 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 07:06:09.681551 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 07:06:09.681594 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"mutex_wait_us":4,"thread_start_us":66,"threads_started":1}
I20260808 07:06:09.681744 2804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:09.677199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47114 (local address 127.13.207.2:39529)
0808 07:06:09.677443 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:09.677449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:09.677611 (+ 162us) server_negotiation.cc:407] Connection header received
0808 07:06:09.677934 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:09.677937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:09.677952 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:09.678029 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:09.678263 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.679042 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.680546 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.681296 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.681386 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:09.681560 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:09.681592 (+ 32us) server_negotiation.cc:299] Negotiation successful
0808 07:06:09.681636 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":126,"threads_started":1}
I20260808 07:06:09.682075 2802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:09.677012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:43875 (local address 127.0.0.1:33474)
0808 07:06:09.677454 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:09.677501 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:09.677590 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:09.677916 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:09.677919 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:09.677922 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:09.678001 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:09.678003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.679393 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.679394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:09.680458 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:09.680460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.681558 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.681560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:09.681643 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:09.681647 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:09.681703 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 07:06:09.681754 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":68,"threads_started":1}
I20260808 07:06:09.682446 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:09.677284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33474 (local address 127.13.207.3:43875)
0808 07:06:09.677526 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:09.677531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:09.677582 (+ 51us) server_negotiation.cc:407] Connection header received
0808 07:06:09.677689 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:09.677692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:09.677707 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:09.677815 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:09.678091 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.679308 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.680542 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.681335 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.681396 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:09.681502 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:09.682310 (+ 808us) server_negotiation.cc:299] Negotiation successful
0808 07:06:09.682356 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":114,"threads_started":1}
I20260808 07:06:09.682529 2582 tablet_service.cc:1467] Processing CreateTablet for tablet 3b8e159ea3454201afe64db60a73618b (DEFAULT_TABLE table=test-table [id=c706513a8be34088b497cd0f79f2ff97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:09.682993 2582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b8e159ea3454201afe64db60a73618b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:09.683097 2801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:09.677053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:39783 (local address 127.0.0.1:41630)
0808 07:06:09.677706 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:09.677715 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:09.677770 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:09.678020 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:09.678020 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:09.678022 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:09.678073 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:09.678074 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.680547 (+ 2473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.680548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:09.681301 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:09.681308 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.682825 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.682826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:09.682893 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:09.682898 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:06:09.682971 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 07:06:09.683031 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":84,"threads_started":1}
I20260808 07:06:09.683115 2805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:09.677298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41630 (local address 127.13.207.1:39783)
0808 07:06:09.677656 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:09.677660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:09.677765 (+ 105us) server_negotiation.cc:407] Connection header received
0808 07:06:09.677865 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:09.677867 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:09.677883 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:09.677953 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:09.678155 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.679304 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.681882 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:09.682731 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:09.682804 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:09.682890 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:09.682984 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 07:06:09.683023 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":83,"threads_started":1}
I20260808 07:06:09.683141 2723 tablet_service.cc:1467] Processing CreateTablet for tablet 3b8e159ea3454201afe64db60a73618b (DEFAULT_TABLE table=test-table [id=c706513a8be34088b497cd0f79f2ff97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:09.683441 2723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b8e159ea3454201afe64db60a73618b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:09.685693 2808 tablet_bootstrap.cc:492] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea: Bootstrap starting.
I20260808 07:06:09.685822 2809 tablet_bootstrap.cc:492] T 3b8e159ea3454201afe64db60a73618b P 2ce21617b0bd4edda7a629250f4173ed: Bootstrap starting.
I20260808 07:06:09.686431 2808 tablet_bootstrap.cc:654] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:09.686530 2809 tablet_bootstrap.cc:654] T 3b8e159ea3454201afe64db60a73618b P 2ce21617b0bd4edda7a629250f4173ed: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:09.686515 2444 tablet_service.cc:1467] Processing CreateTablet for tablet 3b8e159ea3454201afe64db60a73618b (DEFAULT_TABLE table=test-table [id=c706513a8be34088b497cd0f79f2ff97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:09.686834 2808 log.cc:824] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:09.686905 2444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b8e159ea3454201afe64db60a73618b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:09.686880 2809 log.cc:824] T 3b8e159ea3454201afe64db60a73618b P 2ce21617b0bd4edda7a629250f4173ed: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:09.687592 2809 tablet_bootstrap.cc:492] T 3b8e159ea3454201afe64db60a73618b P 2ce21617b0bd4edda7a629250f4173ed: No bootstrap required, opened a new log
I20260808 07:06:09.687675 2808 tablet_bootstrap.cc:492] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea: No bootstrap required, opened a new log
I20260808 07:06:09.687674 2809 ts_tablet_manager.cc:1397] T 3b8e159ea3454201afe64db60a73618b P 2ce21617b0bd4edda7a629250f4173ed: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:06:09.687750 2808 ts_tablet_manager.cc:1397] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:06:09.689584 2809 raft_consensus.cc:348] T 3b8e159ea3454201afe64db60a73618b P 2ce21617b0bd4edda7a629250f4173ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00514b9406f64ff88ecbbdd3fd0f46ea" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39529 } } peers { permanent_uuid: "141ad6a3f31f4f52b47fce172cf0f54a" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 39783 } } peers { permanent_uuid: "2ce21617b0bd4edda7a629250f4173ed" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43875 } }
I20260808 07:06:09.689742 2809 raft_consensus.cc:374] T 3b8e159ea3454201afe64db60a73618b P 2ce21617b0bd4edda7a629250f4173ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:09.689788 2809 raft_consensus.cc:729] T 3b8e159ea3454201afe64db60a73618b P 2ce21617b0bd4edda7a629250f4173ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ce21617b0bd4edda7a629250f4173ed, State: Initialized, Role: FOLLOWER
I20260808 07:06:09.689904 2809 consensus_queue.cc:260] T 3b8e159ea3454201afe64db60a73618b P 2ce21617b0bd4edda7a629250f4173ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00514b9406f64ff88ecbbdd3fd0f46ea" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39529 } } peers { permanent_uuid: "141ad6a3f31f4f52b47fce172cf0f54a" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 39783 } } peers { permanent_uuid: "2ce21617b0bd4edda7a629250f4173ed" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43875 } }
I20260808 07:06:09.690178 2809 ts_tablet_manager.cc:1428] T 3b8e159ea3454201afe64db60a73618b P 2ce21617b0bd4edda7a629250f4173ed: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 07:06:09.690317 2793 heartbeater.cc:503] Master 127.13.207.62:42169 was elected leader, sending a full tablet report...
I20260808 07:06:09.690446 2808 raft_consensus.cc:348] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00514b9406f64ff88ecbbdd3fd0f46ea" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39529 } } peers { permanent_uuid: "141ad6a3f31f4f52b47fce172cf0f54a" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 39783 } } peers { permanent_uuid: "2ce21617b0bd4edda7a629250f4173ed" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43875 } }
I20260808 07:06:09.690614 2808 raft_consensus.cc:374] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:09.690657 2808 raft_consensus.cc:729] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00514b9406f64ff88ecbbdd3fd0f46ea, State: Initialized, Role: FOLLOWER
I20260808 07:06:09.690707 2813 tablet_bootstrap.cc:492] T 3b8e159ea3454201afe64db60a73618b P 141ad6a3f31f4f52b47fce172cf0f54a: Bootstrap starting.
I20260808 07:06:09.690814 2808 consensus_queue.cc:260] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00514b9406f64ff88ecbbdd3fd0f46ea" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39529 } } peers { permanent_uuid: "141ad6a3f31f4f52b47fce172cf0f54a" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 39783 } } peers { permanent_uuid: "2ce21617b0bd4edda7a629250f4173ed" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43875 } }
I20260808 07:06:09.691097 2808 ts_tablet_manager.cc:1428] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 07:06:09.691210 2652 heartbeater.cc:503] Master 127.13.207.62:42169 was elected leader, sending a full tablet report...
I20260808 07:06:09.691468 2813 tablet_bootstrap.cc:654] T 3b8e159ea3454201afe64db60a73618b P 141ad6a3f31f4f52b47fce172cf0f54a: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:09.691855 2813 log.cc:824] T 3b8e159ea3454201afe64db60a73618b P 141ad6a3f31f4f52b47fce172cf0f54a: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:09.692670 2813 tablet_bootstrap.cc:492] T 3b8e159ea3454201afe64db60a73618b P 141ad6a3f31f4f52b47fce172cf0f54a: No bootstrap required, opened a new log
I20260808 07:06:09.692728 2813 ts_tablet_manager.cc:1397] T 3b8e159ea3454201afe64db60a73618b P 141ad6a3f31f4f52b47fce172cf0f54a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:06:09.694917 2813 raft_consensus.cc:348] T 3b8e159ea3454201afe64db60a73618b P 141ad6a3f31f4f52b47fce172cf0f54a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00514b9406f64ff88ecbbdd3fd0f46ea" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39529 } } peers { permanent_uuid: "141ad6a3f31f4f52b47fce172cf0f54a" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 39783 } } peers { permanent_uuid: "2ce21617b0bd4edda7a629250f4173ed" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43875 } }
I20260808 07:06:09.695060 2813 raft_consensus.cc:374] T 3b8e159ea3454201afe64db60a73618b P 141ad6a3f31f4f52b47fce172cf0f54a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:09.695083 2813 raft_consensus.cc:729] T 3b8e159ea3454201afe64db60a73618b P 141ad6a3f31f4f52b47fce172cf0f54a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 141ad6a3f31f4f52b47fce172cf0f54a, State: Initialized, Role: FOLLOWER
I20260808 07:06:09.695441 2813 consensus_queue.cc:260] T 3b8e159ea3454201afe64db60a73618b P 141ad6a3f31f4f52b47fce172cf0f54a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00514b9406f64ff88ecbbdd3fd0f46ea" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39529 } } peers { permanent_uuid: "141ad6a3f31f4f52b47fce172cf0f54a" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 39783 } } peers { permanent_uuid: "2ce21617b0bd4edda7a629250f4173ed" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43875 } }
I20260808 07:06:09.695814 2813 ts_tablet_manager.cc:1428] T 3b8e159ea3454201afe64db60a73618b P 141ad6a3f31f4f52b47fce172cf0f54a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:06:09.695838 2513 heartbeater.cc:503] Master 127.13.207.62:42169 was elected leader, sending a full tablet report...
W20260808 07:06:09.723863 2653 tablet.cc:2369] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:06:09.825968 2514 tablet.cc:2369] T 3b8e159ea3454201afe64db60a73618b P 141ad6a3f31f4f52b47fce172cf0f54a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:06:09.901376 2794 tablet.cc:2369] T 3b8e159ea3454201afe64db60a73618b P 2ce21617b0bd4edda7a629250f4173ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:06:10.009099 2814 raft_consensus.cc:482] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:10.009313 2814 raft_consensus.cc:504] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00514b9406f64ff88ecbbdd3fd0f46ea" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39529 } } peers { permanent_uuid: "141ad6a3f31f4f52b47fce172cf0f54a" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 39783 } } peers { permanent_uuid: "2ce21617b0bd4edda7a629250f4173ed" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43875 } }
I20260808 07:06:10.009687 2814 leader_election.cc:288] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 141ad6a3f31f4f52b47fce172cf0f54a (127.13.207.1:39783), 2ce21617b0bd4edda7a629250f4173ed (127.13.207.3:43875)
I20260808 07:06:10.015018 2818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:10.010179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:39783 (local address 127.13.207.2:33693)
0808 07:06:10.010343 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:10.010355 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:10.010427 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:10.010758 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:10.010759 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:10.010765 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:10.010862 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:10.010865 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:10.012377 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:10.012379 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:10.013466 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:10.013470 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:10.014742 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:10.014744 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:10.014835 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:10.014840 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:06:10.014897 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 07:06:10.014937 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":89,"threads_started":1}
I20260808 07:06:10.015486 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:10.010254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:60611 (local address 127.13.207.3:43875)
0808 07:06:10.010329 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:10.010332 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:10.010726 (+ 394us) server_negotiation.cc:407] Connection header received
0808 07:06:10.010778 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:10.010778 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:10.010789 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:10.010877 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:10.011297 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:10.012014 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:10.014093 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:10.014519 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:10.014768 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:10.014778 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:10.015371 (+ 593us) server_negotiation.cc:299] Negotiation successful
0808 07:06:10.015430 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 07:06:10.015439 2805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:10.010272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.207.2:33693 (local address 127.13.207.1:39783)
0808 07:06:10.010399 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:10.010401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:10.010413 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:06:10.010583 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:10.010585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:10.010599 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:10.010663 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:10.011164 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:10.012148 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:10.013682 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:10.014503 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:10.014585 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:10.014594 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:10.015106 (+ 512us) server_negotiation.cc:299] Negotiation successful
0808 07:06:10.015244 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260808 07:06:10.016086 2817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:10.010179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.3:43875 (local address 127.13.207.2:60611)
0808 07:06:10.010502 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:10.010514 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:10.010620 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:10.011010 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:10.011011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:10.011014 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:10.011091 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:10.011094 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:10.012106 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:10.012107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:10.013891 (+ 1784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:10.013895 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:10.014788 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:10.014789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:10.015124 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:10.015130 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:06:10.015389 (+ 259us) client_negotiation.cc:240] Negotiation successful
0808 07:06:10.015487 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":25,"thread_start_us":68,"threads_started":1}
I20260808 07:06:10.016109 2464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b8e159ea3454201afe64db60a73618b" candidate_uuid: "00514b9406f64ff88ecbbdd3fd0f46ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "141ad6a3f31f4f52b47fce172cf0f54a" is_pre_election: true
I20260808 07:06:10.016345 2464 raft_consensus.cc:2454] T 3b8e159ea3454201afe64db60a73618b P 141ad6a3f31f4f52b47fce172cf0f54a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00514b9406f64ff88ecbbdd3fd0f46ea in term 0.
I20260808 07:06:10.017079 2533 leader_election.cc:302] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00514b9406f64ff88ecbbdd3fd0f46ea, 141ad6a3f31f4f52b47fce172cf0f54a; no voters:
I20260808 07:06:10.017078 2744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b8e159ea3454201afe64db60a73618b" candidate_uuid: "00514b9406f64ff88ecbbdd3fd0f46ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ce21617b0bd4edda7a629250f4173ed" is_pre_election: true
I20260808 07:06:10.017262 2744 raft_consensus.cc:2454] T 3b8e159ea3454201afe64db60a73618b P 2ce21617b0bd4edda7a629250f4173ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00514b9406f64ff88ecbbdd3fd0f46ea in term 0.
I20260808 07:06:10.017452 2814 raft_consensus.cc:2781] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:10.017583 2814 raft_consensus.cc:482] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:10.017654 2814 raft_consensus.cc:3037] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:10.018532 2814 raft_consensus.cc:504] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00514b9406f64ff88ecbbdd3fd0f46ea" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39529 } } peers { permanent_uuid: "141ad6a3f31f4f52b47fce172cf0f54a" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 39783 } } peers { permanent_uuid: "2ce21617b0bd4edda7a629250f4173ed" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43875 } }
I20260808 07:06:10.018743 2814 leader_election.cc:288] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea [CANDIDATE]: Term 1 election: Requested vote from peers 141ad6a3f31f4f52b47fce172cf0f54a (127.13.207.1:39783), 2ce21617b0bd4edda7a629250f4173ed (127.13.207.3:43875)
I20260808 07:06:10.019085 2744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b8e159ea3454201afe64db60a73618b" candidate_uuid: "00514b9406f64ff88ecbbdd3fd0f46ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ce21617b0bd4edda7a629250f4173ed"
I20260808 07:06:10.019313 2744 raft_consensus.cc:3037] T 3b8e159ea3454201afe64db60a73618b P 2ce21617b0bd4edda7a629250f4173ed [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:10.019330 2464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b8e159ea3454201afe64db60a73618b" candidate_uuid: "00514b9406f64ff88ecbbdd3fd0f46ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "141ad6a3f31f4f52b47fce172cf0f54a"
I20260808 07:06:10.019464 2464 raft_consensus.cc:3037] T 3b8e159ea3454201afe64db60a73618b P 141ad6a3f31f4f52b47fce172cf0f54a [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:10.020505 2464 raft_consensus.cc:2454] T 3b8e159ea3454201afe64db60a73618b P 141ad6a3f31f4f52b47fce172cf0f54a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00514b9406f64ff88ecbbdd3fd0f46ea in term 1.
I20260808 07:06:10.020874 2744 raft_consensus.cc:2454] T 3b8e159ea3454201afe64db60a73618b P 2ce21617b0bd4edda7a629250f4173ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00514b9406f64ff88ecbbdd3fd0f46ea in term 1.
I20260808 07:06:10.021096 2533 leader_election.cc:302] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00514b9406f64ff88ecbbdd3fd0f46ea, 141ad6a3f31f4f52b47fce172cf0f54a; no voters:
I20260808 07:06:10.021207 2814 raft_consensus.cc:2781] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:10.021397 2814 raft_consensus.cc:686] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea [term 1 LEADER]: Becoming Leader. State: Replica: 00514b9406f64ff88ecbbdd3fd0f46ea, State: Running, Role: LEADER
I20260808 07:06:10.021522 2814 consensus_queue.cc:237] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00514b9406f64ff88ecbbdd3fd0f46ea" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39529 } } peers { permanent_uuid: "141ad6a3f31f4f52b47fce172cf0f54a" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 39783 } } peers { permanent_uuid: "2ce21617b0bd4edda7a629250f4173ed" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43875 } }
I20260808 07:06:10.022580 2311 catalog_manager.cc:5495] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 00514b9406f64ff88ecbbdd3fd0f46ea (127.13.207.2). New cstate: current_term: 1 leader_uuid: "00514b9406f64ff88ecbbdd3fd0f46ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00514b9406f64ff88ecbbdd3fd0f46ea" member_type: VOTER last_known_addr { host: "127.13.207.2" port: 39529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "141ad6a3f31f4f52b47fce172cf0f54a" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 39783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ce21617b0bd4edda7a629250f4173ed" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43875 } health_report { overall_health: UNKNOWN } } }
I20260808 07:06:10.047333 2804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:10.044245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47116 (local address 127.13.207.2:39529)
0808 07:06:10.044305 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:10.044308 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:10.044322 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:06:10.044389 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:10.044390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:10.044422 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:10.044500 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:10.044715 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:10.045329 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:10.046247 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:10.046466 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:10.046562 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:10.046800 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:10.046915 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:10.046991 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:10.047188 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 07:06:10.047238 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 07:06:10.049633 2744 raft_consensus.cc:1261] T 3b8e159ea3454201afe64db60a73618b P 2ce21617b0bd4edda7a629250f4173ed [term 1 FOLLOWER]: Refusing update from remote peer 00514b9406f64ff88ecbbdd3fd0f46ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:10.049633 2464 raft_consensus.cc:1261] T 3b8e159ea3454201afe64db60a73618b P 141ad6a3f31f4f52b47fce172cf0f54a [term 1 FOLLOWER]: Refusing update from remote peer 00514b9406f64ff88ecbbdd3fd0f46ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:10.050025 2814 consensus_queue.cc:1035] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ce21617b0bd4edda7a629250f4173ed" member_type: VOTER last_known_addr { host: "127.13.207.3" port: 43875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:10.050319 2819 consensus_queue.cc:1035] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "141ad6a3f31f4f52b47fce172cf0f54a" member_type: VOTER last_known_addr { host: "127.13.207.1" port: 39783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:10.054167 2822 mvcc.cc:205] Tried to move back new op lower bound from 7316163666120183808 to 7316163666008997888. Current Snapshot: MvccSnapshot[applied={T|T < 7316163666120183808}]
I20260808 07:06:10.054790 2824 mvcc.cc:205] Tried to move back new op lower bound from 7316163666120183808 to 7316163666008997888. Current Snapshot: MvccSnapshot[applied={T|T < 7316163666120183808}]
I20260808 07:06:10.055374 2823 mvcc.cc:205] Tried to move back new op lower bound from 7316163666120183808 to 7316163666008997888. Current Snapshot: MvccSnapshot[applied={T|T < 7316163666120183808 or (T in {7316163666120183808})}]
I20260808 07:06:10.057185 2312 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:10.057291 2312 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:10.058394 2312 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76852935a51a444789a95963588edbf1: Sending DeleteTablet for 3 replicas of tablet 3b8e159ea3454201afe64db60a73618b
I20260808 07:06:10.058945 2444 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b8e159ea3454201afe64db60a73618b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:10 UTC) from {username='slave'} at 127.0.0.1:41630
I20260808 07:06:10.058990 2723 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b8e159ea3454201afe64db60a73618b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:10 UTC) from {username='slave'} at 127.0.0.1:33474
I20260808 07:06:10.059059 2582 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b8e159ea3454201afe64db60a73618b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:10 UTC) from {username='slave'} at 127.0.0.1:47114
I20260808 07:06:10.059424 2832 tablet_replica.cc:330] T 3b8e159ea3454201afe64db60a73618b P 141ad6a3f31f4f52b47fce172cf0f54a: stopping tablet replica
I20260808 07:06:10.059441 2833 tablet_replica.cc:330] T 3b8e159ea3454201afe64db60a73618b P 2ce21617b0bd4edda7a629250f4173ed: stopping tablet replica
I20260808 07:06:10.059566 2832 raft_consensus.cc:2229] T 3b8e159ea3454201afe64db60a73618b P 141ad6a3f31f4f52b47fce172cf0f54a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:10.059648 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 2375
I20260808 07:06:10.059731 2832 raft_consensus.cc:2258] T 3b8e159ea3454201afe64db60a73618b P 141ad6a3f31f4f52b47fce172cf0f54a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:10.061232 2833 raft_consensus.cc:2229] T 3b8e159ea3454201afe64db60a73618b P 2ce21617b0bd4edda7a629250f4173ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:10.061546 2833 raft_consensus.cc:2258] T 3b8e159ea3454201afe64db60a73618b P 2ce21617b0bd4edda7a629250f4173ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:10.059329 2834 tablet_replica.cc:330] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea: stopping tablet replica
I20260808 07:06:10.066293 2834 raft_consensus.cc:2229] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:06:10.066463 2834 raft_consensus.cc:2258] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:10.067071 2834 ts_tablet_manager.cc:1905] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:10.067723 2833 ts_tablet_manager.cc:1905] T 3b8e159ea3454201afe64db60a73618b P 2ce21617b0bd4edda7a629250f4173ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:10.070173 2834 ts_tablet_manager.cc:1918] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:10.070268 2833 ts_tablet_manager.cc:1918] T 3b8e159ea3454201afe64db60a73618b P 2ce21617b0bd4edda7a629250f4173ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:10.070310 2834 log.cc:1196] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-1/wal/wals/3b8e159ea3454201afe64db60a73618b
I20260808 07:06:10.070361 2833 log.cc:1196] T 3b8e159ea3454201afe64db60a73618b P 2ce21617b0bd4edda7a629250f4173ed: Deleting WAL directory at /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172493058068-14140-0/minicluster-data/ts-2/wal/wals/3b8e159ea3454201afe64db60a73618b
I20260808 07:06:10.070729 2833 ts_tablet_manager.cc:1939] T 3b8e159ea3454201afe64db60a73618b P 2ce21617b0bd4edda7a629250f4173ed: Deleting consensus metadata
I20260808 07:06:10.070729 2834 ts_tablet_manager.cc:1939] T 3b8e159ea3454201afe64db60a73618b P 00514b9406f64ff88ecbbdd3fd0f46ea: Deleting consensus metadata
I20260808 07:06:10.071408 2298 catalog_manager.cc:4841] TS 00514b9406f64ff88ecbbdd3fd0f46ea (127.13.207.2:39529): tablet 3b8e159ea3454201afe64db60a73618b (table test-table [id=c706513a8be34088b497cd0f79f2ff97]) successfully deleted
I20260808 07:06:10.071683 2296 catalog_manager.cc:4841] TS 2ce21617b0bd4edda7a629250f4173ed (127.13.207.3:43875): tablet 3b8e159ea3454201afe64db60a73618b (table test-table [id=c706513a8be34088b497cd0f79f2ff97]) successfully deleted
W20260808 07:06:10.074434 2297 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.207.1:39783 (error 108)
I20260808 07:06:10.074606 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 2516
I20260808 07:06:10.076431 2803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:10.076253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.1:39783 (local address 127.0.0.1:41640)
0808 07:06:10.076326 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:10.076393 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.1:39783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 07:06:10.076690 2297 catalog_manager.cc:4584] TS 141ad6a3f31f4f52b47fce172cf0f54a (127.13.207.1:39783): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3b8e159ea3454201afe64db60a73618b: Network error: Client connection negotiation failed: client connection to 127.13.207.1:39783: connect: Connection refused (error 111)
I20260808 07:06:10.084739 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 2655
I20260808 07:06:10.096479 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 2279
2026-08-08T07:06:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1107 ms)
[----------] 6 tests from EncryptionPolicyTest (10631 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:10 dist-test-slave-8tdl krb5kdc[2839](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:10 dist-test-slave-8tdl krb5kdc[2839](info): set up 2 sockets
Aug 08 07:06:10 dist-test-slave-8tdl krb5kdc[2839](info): commencing operation
krb5kdc: starting...
W20260808 07:06:11.284750 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:06:11 dist-test-slave-8tdl krb5kdc[2839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:06:11.454471 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45543
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:11.547492 2854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:11.548067 2854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:11.548110 2854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:11.549829 2854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:11.549922 2854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:11.549939 2854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:11.550127 2854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:11.550153 2854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:11.552170 2854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45543
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:11.552433 2854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:11.552742 2854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:11.558745 2859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:11.558770 2860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:11.558745 2862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:11.558952 2854 server_base.cc:1034] running on GCE node
I20260808 07:06:11.559571 2854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:11.559948 2854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:11.561170 2854 hybrid_clock.cc:648] HybridClock initialized: now 1786172771561145 us; error 41 us; skew 500 ppm
Aug 08 07:06:11 dist-test-slave-8tdl krb5kdc[2839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172771, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:11.564611 2854 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:06:11.565171 2854 webserver.cc:460] Webserver started at http://127.0.0.1:39681/ using document root <none> and password file <none>
I20260808 07:06:11.565435 2854 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:11.565516 2854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:11.565739 2854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:11.567116 2854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "3e518bf9c5fb4c96aeb1ee959834f455"
format_stamp: "Formatted at 2026-08-08 07:06:11 on dist-test-slave-8tdl"
server_key: "97eef6088dbafde67d2e3ec5f2dce536"
server_key_iv: "453cb38115de2fd652ebc6f6712fe205"
server_key_version: "encryptionkey@0"
I20260808 07:06:11.567534 2854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "3e518bf9c5fb4c96aeb1ee959834f455"
format_stamp: "Formatted at 2026-08-08 07:06:11 on dist-test-slave-8tdl"
server_key: "97eef6088dbafde67d2e3ec5f2dce536"
server_key_iv: "453cb38115de2fd652ebc6f6712fe205"
server_key_version: "encryptionkey@0"
I20260808 07:06:11.569137 2854 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:06:11.570472 2869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:11.570816 2854 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:11.570927 2854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "3e518bf9c5fb4c96aeb1ee959834f455"
format_stamp: "Formatted at 2026-08-08 07:06:11 on dist-test-slave-8tdl"
server_key: "97eef6088dbafde67d2e3ec5f2dce536"
server_key_iv: "453cb38115de2fd652ebc6f6712fe205"
server_key_version: "encryptionkey@0"
I20260808 07:06:11.571038 2854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:11.590232 2854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:11.592872 2854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:11.593067 2854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:11.598212 2854 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45543
I20260808 07:06:11.598286 2924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45543 every 8 connection(s)
I20260808 07:06:11.598779 2854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:11.599838 2925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:11.602344 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 2854
I20260808 07:06:11.602348 2925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e518bf9c5fb4c96aeb1ee959834f455: Bootstrap starting.
I20260808 07:06:11.602460 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:11.602743 14140 external_mini_cluster.cc:1428] Setting key bdc4dc22a790d7cc570414efd8f6cf1c
I20260808 07:06:11.603247 2925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e518bf9c5fb4c96aeb1ee959834f455: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:11.603657 2925 log.cc:824] T 00000000000000000000000000000000 P 3e518bf9c5fb4c96aeb1ee959834f455: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:11.604568 2925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e518bf9c5fb4c96aeb1ee959834f455: No bootstrap required, opened a new log
I20260808 07:06:11.607066 2925 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e518bf9c5fb4c96aeb1ee959834f455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e518bf9c5fb4c96aeb1ee959834f455" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45543 } }
I20260808 07:06:11.607271 2925 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e518bf9c5fb4c96aeb1ee959834f455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:11.607307 2925 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e518bf9c5fb4c96aeb1ee959834f455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e518bf9c5fb4c96aeb1ee959834f455, State: Initialized, Role: FOLLOWER
I20260808 07:06:11.607481 2925 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e518bf9c5fb4c96aeb1ee959834f455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e518bf9c5fb4c96aeb1ee959834f455" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45543 } }
I20260808 07:06:11.607563 2925 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e518bf9c5fb4c96aeb1ee959834f455 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:11.607599 2925 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e518bf9c5fb4c96aeb1ee959834f455 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:11.607651 2925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e518bf9c5fb4c96aeb1ee959834f455 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:11.608359 2925 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e518bf9c5fb4c96aeb1ee959834f455 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e518bf9c5fb4c96aeb1ee959834f455" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45543 } }
I20260808 07:06:11.608511 2925 leader_election.cc:302] T 00000000000000000000000000000000 P 3e518bf9c5fb4c96aeb1ee959834f455 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e518bf9c5fb4c96aeb1ee959834f455; no voters:
I20260808 07:06:11.608713 2925 leader_election.cc:288] T 00000000000000000000000000000000 P 3e518bf9c5fb4c96aeb1ee959834f455 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:11.608788 2931 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e518bf9c5fb4c96aeb1ee959834f455 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:11.609006 2931 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e518bf9c5fb4c96aeb1ee959834f455 [term 1 LEADER]: Becoming Leader. State: Replica: 3e518bf9c5fb4c96aeb1ee959834f455, State: Running, Role: LEADER
I20260808 07:06:11.609114 2931 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e518bf9c5fb4c96aeb1ee959834f455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e518bf9c5fb4c96aeb1ee959834f455" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45543 } }
I20260808 07:06:11.609479 2925 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e518bf9c5fb4c96aeb1ee959834f455 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:11.609520 2932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e518bf9c5fb4c96aeb1ee959834f455 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e518bf9c5fb4c96aeb1ee959834f455" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e518bf9c5fb4c96aeb1ee959834f455" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45543 } } }
I20260808 07:06:11.609601 2932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e518bf9c5fb4c96aeb1ee959834f455 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:11.609851 2933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e518bf9c5fb4c96aeb1ee959834f455 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e518bf9c5fb4c96aeb1ee959834f455. Latest consensus state: current_term: 1 leader_uuid: "3e518bf9c5fb4c96aeb1ee959834f455" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e518bf9c5fb4c96aeb1ee959834f455" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45543 } } }
I20260808 07:06:11.609969 2933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e518bf9c5fb4c96aeb1ee959834f455 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:11.610386 2935 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:11.610999 2935 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 07:06:11 dist-test-slave-8tdl krb5kdc[2839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 07:06:11.612825 2935 catalog_manager.cc:1347] Generated new cluster ID: 0f3a81d7388c459687ee085dd4937d77
I20260808 07:06:11.612885 2935 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:11.616570 2929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:11.603542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59383 (local address 127.0.0.1:45543)
0808 07:06:11.603763 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:11.603773 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:11.603829 (+ 56us) server_negotiation.cc:407] Connection header received
0808 07:06:11.604468 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:11.604476 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:11.604530 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:11.604709 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:11.605680 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:11.606670 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:11.607827 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:11.608377 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:11.608497 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:11.612217 (+ 3720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:11.612222 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:11.612222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:11.612236 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:11.614505 (+ 2269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:11.615109 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:11.615112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:11.615112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:11.615231 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:11.615541 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:11.615541 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:11.615542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:11.615866 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:11.616053 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:11.616193 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 07:06:11.616300 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":8,"server-negotiator.queue_time_us":150,"thread_start_us":107,"threads_started":1}
I20260808 07:06:11.617859 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:11.631565 2929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:11.619548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42161 (local address 127.0.0.1:45543)
0808 07:06:11.619588 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:11.619590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:11.622933 (+ 3343us) server_negotiation.cc:407] Connection header received
0808 07:06:11.623024 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:11.623026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:11.623040 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:11.623069 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:11.623845 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:11.624738 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:11.625846 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:11.626194 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:11.626275 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:11.628187 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:11.628189 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:11.628189 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:11.628199 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:11.630049 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:11.630550 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:11.630551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:11.630552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:11.630621 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:11.630939 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:11.630940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:11.630940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:11.631098 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:11.631214 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:11.631373 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 07:06:11.631450 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:06:11.649220 2935 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:11.650246 2935 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:11.654727 2935 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e518bf9c5fb4c96aeb1ee959834f455: Generated new TSK 0
I20260808 07:06:11.654982 2935 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:11.663726 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 2854
W20260808 07:06:11.670039 2836 connection.cc:535] client connection to 127.0.0.1:45543 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45543): Connection reset by peer (error 104)
2026-08-08T07:06:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1563 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:11 dist-test-slave-8tdl krb5kdc[2958](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:11 dist-test-slave-8tdl krb5kdc[2958](info): set up 2 sockets
Aug 08 07:06:11 dist-test-slave-8tdl krb5kdc[2958](info): commencing operation
krb5kdc: starting...
W20260808 07:06:12.811800 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:06:12 dist-test-slave-8tdl krb5kdc[2958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:06:12.963253 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34625
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:13.056284 2972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:13.056583 2972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:13.056631 2972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:13.057940 2972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:13.058039 2972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:13.058065 2972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:13.058089 2972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:13.058117 2972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:13.060105 2972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34625
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:13.060479 2972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:13.060881 2972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:13.066628 2978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:13.066617 2977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:13.066861 2972 server_base.cc:1034] running on GCE node
W20260808 07:06:13.066633 2980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:13.067271 2972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:13.067641 2972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:13.068843 2972 hybrid_clock.cc:648] HybridClock initialized: now 1786172773068779 us; error 81 us; skew 500 ppm
Aug 08 07:06:13 dist-test-slave-8tdl krb5kdc[2958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172773, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:13.076915 2972 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:06:13.077817 2972 webserver.cc:460] Webserver started at http://127.0.0.1:35075/ using document root <none> and password file <none>
I20260808 07:06:13.078109 2972 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:13.078182 2972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:13.078337 2972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:13.079597 2972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "500db20e3c2b47b1893a043fedca3039"
format_stamp: "Formatted at 2026-08-08 07:06:13 on dist-test-slave-8tdl"
server_key: "e8477bc2b16765cf0b9867e440c016ce"
server_key_iv: "f137f85e2a7b08d878bece048ba1c0b7"
server_key_version: "encryptionkey@0"
I20260808 07:06:13.080003 2972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "500db20e3c2b47b1893a043fedca3039"
format_stamp: "Formatted at 2026-08-08 07:06:13 on dist-test-slave-8tdl"
server_key: "e8477bc2b16765cf0b9867e440c016ce"
server_key_iv: "f137f85e2a7b08d878bece048ba1c0b7"
server_key_version: "encryptionkey@0"
I20260808 07:06:13.082250 2972 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 07:06:13.083451 2987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:13.083822 2972 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:06:13.084092 2972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "500db20e3c2b47b1893a043fedca3039"
format_stamp: "Formatted at 2026-08-08 07:06:13 on dist-test-slave-8tdl"
server_key: "e8477bc2b16765cf0b9867e440c016ce"
server_key_iv: "f137f85e2a7b08d878bece048ba1c0b7"
server_key_version: "encryptionkey@0"
I20260808 07:06:13.084195 2972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:13.107846 2972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:13.110654 2972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:13.110896 2972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:13.117285 2972 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34625
I20260808 07:06:13.117306 3043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34625 every 8 connection(s)
I20260808 07:06:13.117741 2972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:13.118769 3044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:13.120913 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 2972
I20260808 07:06:13.121073 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:13.121296 14140 external_mini_cluster.cc:1428] Setting key c26d51e89b4d4fe521b24dce6aea3ce4
I20260808 07:06:13.122174 3044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039: Bootstrap starting.
I20260808 07:06:13.122923 3044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:13.123790 3044 log.cc:824] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:13.124903 3044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039: No bootstrap required, opened a new log
I20260808 07:06:13.127470 3044 raft_consensus.cc:348] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "500db20e3c2b47b1893a043fedca3039" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34625 } }
I20260808 07:06:13.127702 3044 raft_consensus.cc:374] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:13.127784 3044 raft_consensus.cc:729] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500db20e3c2b47b1893a043fedca3039, State: Initialized, Role: FOLLOWER
I20260808 07:06:13.127787 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:13.122144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34651 (local address 127.0.0.1:34625)
0808 07:06:13.122359 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:13.122368 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:13.122642 (+ 274us) server_negotiation.cc:407] Connection header received
0808 07:06:13.123251 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:13.123259 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:13.123315 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:13.123475 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:13.123739 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:13.124710 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:13.125573 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:13.126054 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:13.126198 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:13.127078 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:13.127083 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:13.127083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:13.127268 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:13.127307 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:13.127320 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:13.127420 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:13.127527 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 07:06:13.127560 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":109,"threads_started":1}
I20260808 07:06:13.128180 3044 consensus_queue.cc:260] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "500db20e3c2b47b1893a043fedca3039" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34625 } }
I20260808 07:06:13.128413 3044 raft_consensus.cc:388] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:13.128469 3044 raft_consensus.cc:482] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:13.128516 3044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:13.129300 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:13.129314 3044 raft_consensus.cc:504] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "500db20e3c2b47b1893a043fedca3039" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34625 } }
I20260808 07:06:13.129439 3044 leader_election.cc:302] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 500db20e3c2b47b1893a043fedca3039; no voters:
I20260808 07:06:13.129665 3044 leader_election.cc:288] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:13.129724 3050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:13.129923 3050 raft_consensus.cc:686] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039 [term 1 LEADER]: Becoming Leader. State: Replica: 500db20e3c2b47b1893a043fedca3039, State: Running, Role: LEADER
I20260808 07:06:13.129982 3044 sys_catalog.cc:564] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:13.130187 3050 consensus_queue.cc:237] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "500db20e3c2b47b1893a043fedca3039" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34625 } }
I20260808 07:06:13.132124 3051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "500db20e3c2b47b1893a043fedca3039" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "500db20e3c2b47b1893a043fedca3039" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34625 } } }
I20260808 07:06:13.132225 3051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039 [sys.catalog]: This master's current role is: LEADER
W20260808 07:06:13.132491 3070 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 07:06:13.132454 3052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 500db20e3c2b47b1893a043fedca3039. Latest consensus state: current_term: 1 leader_uuid: "500db20e3c2b47b1893a043fedca3039" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "500db20e3c2b47b1893a043fedca3039" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34625 } } }
I20260808 07:06:13.132638 3052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039 [sys.catalog]: This master's current role is: LEADER
W20260808 07:06:13.132700 3070 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:06:13.132773 3073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:13.132995 3073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:13.135802 3073 catalog_manager.cc:1347] Generated new cluster ID: f0c98844cb1b4d0a86ee7aaf72220857
I20260808 07:06:13.135867 3073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:13.136622 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:13.131932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45693 (local address 127.0.0.1:34625)
0808 07:06:13.131977 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:13.131980 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:13.132167 (+ 187us) server_negotiation.cc:407] Connection header received
0808 07:06:13.132247 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:13.132248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:13.132261 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:13.132331 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:13.132559 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:13.133237 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:13.134204 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:13.134524 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:13.134622 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:13.136323 (+ 1701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:13.136325 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:13.136325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:13.136335 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:13.136356 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:13.136362 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:13.136437 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:13.136506 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 07:06:13.136519 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:06:13.152388 3073 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:13.153338 3073 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:13.161742 3073 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 500db20e3c2b47b1893a043fedca3039: Generated new TSK 0
I20260808 07:06:13.162065 3073 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:13.170915 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 2972
W20260808 07:06:13.177636 2956 connection.cc:535] client connection to 127.0.0.1:34625 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34625): Connection reset by peer (error 104)
2026-08-08T07:06:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1508 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:13 dist-test-slave-8tdl krb5kdc[3077](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:13 dist-test-slave-8tdl krb5kdc[3077](info): set up 2 sockets
Aug 08 07:06:13 dist-test-slave-8tdl krb5kdc[3077](info): commencing operation
krb5kdc: starting...
W20260808 07:06:14.323012 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:06:14 dist-test-slave-8tdl krb5kdc[3077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:06:14.471921 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40925
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:14.564138 3091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:14.564414 3091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:14.564456 3091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:14.565904 3091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:14.565984 3091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:14.566051 3091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:14.566078 3091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:14.566112 3091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:14.567895 3091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40925
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:14.568212 3091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:14.568488 3091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:14.573997 3097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:14.573997 3099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:14.574112 3096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:14.574493 3091 server_base.cc:1034] running on GCE node
I20260808 07:06:14.574826 3091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:14.575127 3091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:14.576328 3091 hybrid_clock.cc:648] HybridClock initialized: now 1786172774576314 us; error 35 us; skew 500 ppm
Aug 08 07:06:14 dist-test-slave-8tdl krb5kdc[3077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172774, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:14.579391 3091 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:06:14.580003 3091 webserver.cc:460] Webserver started at http://127.0.0.1:45035/ using document root <none> and password file <none>
I20260808 07:06:14.580245 3091 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:14.580317 3091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:14.580482 3091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:14.582109 3091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "b3838236263f450398cfe392e28c9dd0"
format_stamp: "Formatted at 2026-08-08 07:06:14 on dist-test-slave-8tdl"
server_key: "3b1515c6a75268eed3b910e0e93e35d0"
server_key_iv: "ba48454e83c6489e301b9894ed68b3b1"
server_key_version: "encryptionkey@0"
I20260808 07:06:14.582558 3091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "b3838236263f450398cfe392e28c9dd0"
format_stamp: "Formatted at 2026-08-08 07:06:14 on dist-test-slave-8tdl"
server_key: "3b1515c6a75268eed3b910e0e93e35d0"
server_key_iv: "ba48454e83c6489e301b9894ed68b3b1"
server_key_version: "encryptionkey@0"
I20260808 07:06:14.584259 3091 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:14.585198 3106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:14.585608 3091 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:14.585755 3091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "b3838236263f450398cfe392e28c9dd0"
format_stamp: "Formatted at 2026-08-08 07:06:14 on dist-test-slave-8tdl"
server_key: "3b1515c6a75268eed3b910e0e93e35d0"
server_key_iv: "ba48454e83c6489e301b9894ed68b3b1"
server_key_version: "encryptionkey@0"
I20260808 07:06:14.585925 3091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:14.589804 3091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:14.592540 3091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:14.592756 3091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:14.597970 3091 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40925
I20260808 07:06:14.598042 3160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40925 every 8 connection(s)
I20260808 07:06:14.598428 3091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:14.598927 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 3091
I20260808 07:06:14.599035 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:14.599269 14140 external_mini_cluster.cc:1428] Setting key 113f3fec8d7842c4f9933acac3141ffa
I20260808 07:06:14.599852 3161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:14.602030 3164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:14.599864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57145 (local address 127.0.0.1:40925)
0808 07:06:14.600065 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:14.600082 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:14.600109 (+ 27us) server_negotiation.cc:407] Connection header received
0808 07:06:14.600575 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:14.600582 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:14.600613 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:14.600749 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:14.601520 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:14.601523 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:14.601524 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:14.601527 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:14.601576 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:14.601587 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:14.601645 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:14.601745 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 07:06:14.601805 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":104,"threads_started":1}
I20260808 07:06:14.603250 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:14.603691 3161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0: Bootstrap starting.
I20260808 07:06:14.604521 3161 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:14.604928 3161 log.cc:824] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:14.605890 3161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0: No bootstrap required, opened a new log
I20260808 07:06:14.606261 3164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:14.605032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39007 (local address 127.0.0.1:40925)
0808 07:06:14.605094 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:14.605096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:14.605238 (+ 142us) server_negotiation.cc:407] Connection header received
0808 07:06:14.605313 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:14.605314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:14.605319 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:14.605373 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:14.605978 (+ 605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:14.605979 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:14.605980 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:14.605984 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:14.606000 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:14.606004 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:14.606056 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:14.606174 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 07:06:14.606190 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:06:14.608147 3161 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3838236263f450398cfe392e28c9dd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40925 } }
I20260808 07:06:14.608299 3161 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:14.608325 3161 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3838236263f450398cfe392e28c9dd0, State: Initialized, Role: FOLLOWER
I20260808 07:06:14.608458 3161 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3838236263f450398cfe392e28c9dd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40925 } }
I20260808 07:06:14.608553 3161 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:14.608611 3161 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:14.608664 3161 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:14.609288 3161 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3838236263f450398cfe392e28c9dd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40925 } }
I20260808 07:06:14.609380 3161 leader_election.cc:302] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3838236263f450398cfe392e28c9dd0; no voters:
I20260808 07:06:14.609591 3161 leader_election.cc:288] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:14.609714 3171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:14.609936 3161 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:14.609968 3171 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0 [term 1 LEADER]: Becoming Leader. State: Replica: b3838236263f450398cfe392e28c9dd0, State: Running, Role: LEADER
I20260808 07:06:14.610847 3171 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3838236263f450398cfe392e28c9dd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40925 } }
I20260808 07:06:14.611474 3173 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3838236263f450398cfe392e28c9dd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3838236263f450398cfe392e28c9dd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40925 } } }
I20260808 07:06:14.611639 3174 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3838236263f450398cfe392e28c9dd0. Latest consensus state: current_term: 1 leader_uuid: "b3838236263f450398cfe392e28c9dd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3838236263f450398cfe392e28c9dd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40925 } } }
I20260808 07:06:14.611725 3173 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:14.611730 3174 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0 [sys.catalog]: This master's current role is: LEADER
W20260808 07:06:14.612174 3187 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:06:14.612237 3187 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:06:14.612299 3188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:14.612480 3188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:14.616537 3188 catalog_manager.cc:1347] Generated new cluster ID: e27d567256d94368a86af26574e9364c
I20260808 07:06:14.616642 3188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:14.637705 3188 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:14.637944 3188 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:14.644649 3188 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3838236263f450398cfe392e28c9dd0: Generated new TSK 0
I20260808 07:06:14.644917 3188 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:14.652158 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 3091
2026-08-08T07:06:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1485 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:14 dist-test-slave-8tdl krb5kdc[3194](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:14 dist-test-slave-8tdl krb5kdc[3194](info): set up 2 sockets
Aug 08 07:06:14 dist-test-slave-8tdl krb5kdc[3194](info): commencing operation
krb5kdc: starting...
W20260808 07:06:15.915212 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.002s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:06:16 dist-test-slave-8tdl krb5kdc[3194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:06:16.064558 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46787
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:16.161847 3208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:16.162113 3208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:16.162142 3208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:16.163525 3208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:16.163578 3208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:16.163594 3208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:16.163609 3208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:16.163623 3208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:16.165283 3208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:16.165514 3208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:16.165719 3208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:16.171403 3216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:16.171509 3208 server_base.cc:1034] running on GCE node
W20260808 07:06:16.171429 3218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:16.171391 3215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:16.171876 3208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:16.172217 3208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:16.173372 3208 hybrid_clock.cc:648] HybridClock initialized: now 1786172776173350 us; error 47 us; skew 500 ppm
Aug 08 07:06:16 dist-test-slave-8tdl krb5kdc[3194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172776, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:16.176267 3208 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:06:16.176764 3208 webserver.cc:460] Webserver started at http://127.0.0.1:44117/ using document root <none> and password file <none>
I20260808 07:06:16.176960 3208 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:16.177009 3208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:16.177104 3208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:16.178171 3208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "8c30435b8fa241098c0bc8c08c01361c"
format_stamp: "Formatted at 2026-08-08 07:06:16 on dist-test-slave-8tdl"
server_key: "f9e5252f57370fbcca5c78f229e0618c"
server_key_iv: "f0787bcd154e7a16364069fb6ff79754"
server_key_version: "encryptionkey@0"
I20260808 07:06:16.178535 3208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "8c30435b8fa241098c0bc8c08c01361c"
format_stamp: "Formatted at 2026-08-08 07:06:16 on dist-test-slave-8tdl"
server_key: "f9e5252f57370fbcca5c78f229e0618c"
server_key_iv: "f0787bcd154e7a16364069fb6ff79754"
server_key_version: "encryptionkey@0"
I20260808 07:06:16.180119 3208 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:16.181064 3225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:16.181291 3208 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:16.181396 3208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "8c30435b8fa241098c0bc8c08c01361c"
format_stamp: "Formatted at 2026-08-08 07:06:16 on dist-test-slave-8tdl"
server_key: "f9e5252f57370fbcca5c78f229e0618c"
server_key_iv: "f0787bcd154e7a16364069fb6ff79754"
server_key_version: "encryptionkey@0"
I20260808 07:06:16.181473 3208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:16.190655 3208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:16.194221 3208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:16.194379 3208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:16.199370 3208 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46787
I20260808 07:06:16.199398 3282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46787 every 8 connection(s)
I20260808 07:06:16.199810 3208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:16.200794 3283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:16.201192 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 3208
I20260808 07:06:16.201329 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:16.201576 14140 external_mini_cluster.cc:1428] Setting key d3cf0f057d1d2596e07652d803ca4ba6
I20260808 07:06:16.204093 3283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c30435b8fa241098c0bc8c08c01361c: Bootstrap starting.
I20260808 07:06:16.204872 3283 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c30435b8fa241098c0bc8c08c01361c: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:16.205224 3283 log.cc:824] T 00000000000000000000000000000000 P 8c30435b8fa241098c0bc8c08c01361c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:16.206074 3283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c30435b8fa241098c0bc8c08c01361c: No bootstrap required, opened a new log
I20260808 07:06:16.208870 3283 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c30435b8fa241098c0bc8c08c01361c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c30435b8fa241098c0bc8c08c01361c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46787 } }
I20260808 07:06:16.209075 3283 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c30435b8fa241098c0bc8c08c01361c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:16.209123 3283 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c30435b8fa241098c0bc8c08c01361c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c30435b8fa241098c0bc8c08c01361c, State: Initialized, Role: FOLLOWER
I20260808 07:06:16.209277 3283 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c30435b8fa241098c0bc8c08c01361c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c30435b8fa241098c0bc8c08c01361c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46787 } }
I20260808 07:06:16.209383 3283 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c30435b8fa241098c0bc8c08c01361c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:16.209416 3283 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c30435b8fa241098c0bc8c08c01361c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:16.209565 3283 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c30435b8fa241098c0bc8c08c01361c [term 0 FOLLOWER]: Advancing to term 1
Aug 08 07:06:16 dist-test-slave-8tdl krb5kdc[3194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 07:06:16.210501 3283 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c30435b8fa241098c0bc8c08c01361c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c30435b8fa241098c0bc8c08c01361c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46787 } }
I20260808 07:06:16.210807 3283 leader_election.cc:302] T 00000000000000000000000000000000 P 8c30435b8fa241098c0bc8c08c01361c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c30435b8fa241098c0bc8c08c01361c; no voters:
I20260808 07:06:16.211166 3283 leader_election.cc:288] T 00000000000000000000000000000000 P 8c30435b8fa241098c0bc8c08c01361c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:16.211202 3289 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c30435b8fa241098c0bc8c08c01361c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:16.211339 3289 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c30435b8fa241098c0bc8c08c01361c [term 1 LEADER]: Becoming Leader. State: Replica: 8c30435b8fa241098c0bc8c08c01361c, State: Running, Role: LEADER
I20260808 07:06:16.211472 3289 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c30435b8fa241098c0bc8c08c01361c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c30435b8fa241098c0bc8c08c01361c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46787 } }
I20260808 07:06:16.211555 3283 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c30435b8fa241098c0bc8c08c01361c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:16.211829 3289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c30435b8fa241098c0bc8c08c01361c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c30435b8fa241098c0bc8c08c01361c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c30435b8fa241098c0bc8c08c01361c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46787 } } }
I20260808 07:06:16.211903 3289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c30435b8fa241098c0bc8c08c01361c [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:16.211985 3291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c30435b8fa241098c0bc8c08c01361c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c30435b8fa241098c0bc8c08c01361c. Latest consensus state: current_term: 1 leader_uuid: "8c30435b8fa241098c0bc8c08c01361c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c30435b8fa241098c0bc8c08c01361c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46787 } } }
I20260808 07:06:16.212095 3291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c30435b8fa241098c0bc8c08c01361c [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:16.212282 3295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:16.213645 3295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:16.215374 3286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:16.202225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42583 (local address 127.0.0.1:46787)
0808 07:06:16.202455 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:16.202465 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:16.202502 (+ 37us) server_negotiation.cc:407] Connection header received
0808 07:06:16.203128 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:16.203137 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:16.203199 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:16.203367 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:16.204148 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:16.205609 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:16.206846 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:16.207287 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:16.207375 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:16.210854 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:16.210858 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:16.210858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:16.210871 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:16.213250 (+ 2379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:16.213800 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:16.213804 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:16.213804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:16.213909 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:16.214165 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:16.214167 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:16.214167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:16.214455 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:16.214770 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:16.214929 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 07:06:16.215103 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":109,"threads_started":1}
I20260808 07:06:16.216121 3295 catalog_manager.cc:1347] Generated new cluster ID: 9fd1b31f3afe4edf815ff5a9abfb58af
I20260808 07:06:16.216179 3295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:16.216964 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:16.225487 3295 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:16.226354 3295 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:16.229681 3286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:16.218715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54807 (local address 127.0.0.1:46787)
0808 07:06:16.218757 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:16.218759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:16.222745 (+ 3986us) server_negotiation.cc:407] Connection header received
0808 07:06:16.222848 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:16.222850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:16.222862 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:16.222915 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:16.223617 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:16.224298 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:16.225076 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:16.225387 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:16.225484 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:16.226939 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:16.226940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:16.226941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:16.226949 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:16.228423 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:16.228824 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:16.228826 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:16.228826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:16.228890 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:16.229105 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:16.229106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:16.229106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:16.229228 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:16.229345 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:16.229459 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 07:06:16.229555 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":148,"server-negotiator.queue_time_us":21}
I20260808 07:06:16.230420 3295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c30435b8fa241098c0bc8c08c01361c: Generated new TSK 0
I20260808 07:06:16.230640 3295 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:16.233922 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 3208
2026-08-08T07:06:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1576 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:16 dist-test-slave-8tdl krb5kdc[3316](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:16 dist-test-slave-8tdl krb5kdc[3316](info): set up 2 sockets
Aug 08 07:06:16 dist-test-slave-8tdl krb5kdc[3316](info): commencing operation
krb5kdc: starting...
W20260808 07:06:17.381585 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:06:17 dist-test-slave-8tdl krb5kdc[3316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:06:17.527467 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41315
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:17.617430 3329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:17.617689 3329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:17.617719 3329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:17.619206 3329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:17.619272 3329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:17.619289 3329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:17.619305 3329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:17.619319 3329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:17.621505 3329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:17.621848 3329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:17.622139 3329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:17.628154 3335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:17.628163 3337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:17.628225 3334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:17.628701 3329 server_base.cc:1034] running on GCE node
I20260808 07:06:17.629022 3329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:17.629324 3329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:17.630489 3329 hybrid_clock.cc:648] HybridClock initialized: now 1786172777630460 us; error 48 us; skew 500 ppm
Aug 08 07:06:17 dist-test-slave-8tdl krb5kdc[3316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172777, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:17.633432 3329 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:06:17.634001 3329 webserver.cc:460] Webserver started at http://127.0.0.1:44245/ using document root <none> and password file <none>
I20260808 07:06:17.634243 3329 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:17.634291 3329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:17.634474 3329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:17.635798 3329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "b8a53338c03a44dfaf64af88ee5029c3"
format_stamp: "Formatted at 2026-08-08 07:06:17 on dist-test-slave-8tdl"
server_key: "862cd82da2f237b22f2ca067dfff68d0"
server_key_iv: "6c570cbd11eb96b5f4e139bc0129dfa7"
server_key_version: "encryptionkey@0"
I20260808 07:06:17.636206 3329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "b8a53338c03a44dfaf64af88ee5029c3"
format_stamp: "Formatted at 2026-08-08 07:06:17 on dist-test-slave-8tdl"
server_key: "862cd82da2f237b22f2ca067dfff68d0"
server_key_iv: "6c570cbd11eb96b5f4e139bc0129dfa7"
server_key_version: "encryptionkey@0"
I20260808 07:06:17.637967 3329 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 07:06:17.639014 3345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:17.639351 3329 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:17.639549 3329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "b8a53338c03a44dfaf64af88ee5029c3"
format_stamp: "Formatted at 2026-08-08 07:06:17 on dist-test-slave-8tdl"
server_key: "862cd82da2f237b22f2ca067dfff68d0"
server_key_iv: "6c570cbd11eb96b5f4e139bc0129dfa7"
server_key_version: "encryptionkey@0"
I20260808 07:06:17.639643 3329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:17.655320 3329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:17.657747 3329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:17.657994 3329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:17.662824 3329 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41315
I20260808 07:06:17.662871 3403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41315 every 8 connection(s)
I20260808 07:06:17.663259 3329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:17.664274 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 3329
I20260808 07:06:17.664386 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:17.664405 3404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:17.664726 14140 external_mini_cluster.cc:1428] Setting key ac06f20788d81d9805068a4df5d542fa
I20260808 07:06:17.668746 3404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3: Bootstrap starting.
I20260808 07:06:17.669714 3404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:17.670086 3404 log.cc:824] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:17.671211 3404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3: No bootstrap required, opened a new log
I20260808 07:06:17.672802 3407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:17.665440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50455 (local address 127.0.0.1:41315)
0808 07:06:17.665808 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:17.665817 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:17.665846 (+ 29us) server_negotiation.cc:407] Connection header received
0808 07:06:17.666414 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:17.666423 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:17.666490 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:17.666627 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:17.667082 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:17.668383 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:17.669873 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:17.670520 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:17.670939 (+ 419us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:17.671754 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:17.671758 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:17.671759 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:17.671991 (+ 232us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:17.672031 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:17.672043 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:17.672105 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:17.672371 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 07:06:17.672439 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":255,"threads_started":1}
I20260808 07:06:17.674201 3404 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a53338c03a44dfaf64af88ee5029c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41315 } }
I20260808 07:06:17.674396 3404 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:17.674446 3404 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8a53338c03a44dfaf64af88ee5029c3, State: Initialized, Role: FOLLOWER
I20260808 07:06:17.674412 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:17.674621 3404 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a53338c03a44dfaf64af88ee5029c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41315 } }
I20260808 07:06:17.674750 3404 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:17.674798 3404 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:17.674865 3404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:17.675648 3404 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a53338c03a44dfaf64af88ee5029c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41315 } }
I20260808 07:06:17.675777 3404 leader_election.cc:302] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8a53338c03a44dfaf64af88ee5029c3; no voters:
I20260808 07:06:17.675976 3404 leader_election.cc:288] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:17.676039 3413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:17.676265 3413 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3 [term 1 LEADER]: Becoming Leader. State: Replica: b8a53338c03a44dfaf64af88ee5029c3, State: Running, Role: LEADER
I20260808 07:06:17.676329 3404 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:17.676440 3413 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a53338c03a44dfaf64af88ee5029c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41315 } }
I20260808 07:06:17.676836 3416 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8a53338c03a44dfaf64af88ee5029c3. Latest consensus state: current_term: 1 leader_uuid: "b8a53338c03a44dfaf64af88ee5029c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a53338c03a44dfaf64af88ee5029c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41315 } } }
I20260808 07:06:17.676837 3415 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8a53338c03a44dfaf64af88ee5029c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a53338c03a44dfaf64af88ee5029c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41315 } } }
I20260808 07:06:17.677292 3415 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:17.677510 3416 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3 [sys.catalog]: This master's current role is: LEADER
W20260808 07:06:17.678335 3430 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:06:17.678400 3430 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:06:17.678450 3425 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:17.678658 3425 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:17.680488 3425 catalog_manager.cc:1347] Generated new cluster ID: 82169fd4bbad42b983875393a102152b
I20260808 07:06:17.680553 3425 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:17.681762 3407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:17.676040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50085 (local address 127.0.0.1:41315)
0808 07:06:17.676083 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:17.676085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:17.678131 (+ 2046us) server_negotiation.cc:407] Connection header received
0808 07:06:17.678203 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:17.678204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:17.678219 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:17.678291 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:17.678526 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:17.679282 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:17.680261 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:17.680551 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:17.680627 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:17.681433 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:17.681435 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:17.681435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:17.681440 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:17.681455 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:17.681459 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:17.681496 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:17.681660 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 07:06:17.681684 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:06:17.704519 3425 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:17.705411 3425 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:17.711843 3425 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8a53338c03a44dfaf64af88ee5029c3: Generated new TSK 0
I20260808 07:06:17.712096 3425 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:17.721643 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 3329
2026-08-08T07:06:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1490 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:17 dist-test-slave-8tdl krb5kdc[3435](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:17 dist-test-slave-8tdl krb5kdc[3435](info): set up 2 sockets
Aug 08 07:06:17 dist-test-slave-8tdl krb5kdc[3435](info): commencing operation
krb5kdc: starting...
W20260808 07:06:18.845698 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:06:18 dist-test-slave-8tdl krb5kdc[3435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:06:19.003618 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37309
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:19.100957 3448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:19.101202 3448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:19.101229 3448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:19.102450 3448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:19.102495 3448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:19.102510 3448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:19.102523 3448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:19.102535 3448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:19.104384 3448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:19.104655 3448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:19.104856 3448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:19.110833 3453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:19.110834 3456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:19.112528 3448 server_base.cc:1034] running on GCE node
W20260808 07:06:19.112843 3454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:19.113219 3448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:19.113636 3448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:19.114852 3448 hybrid_clock.cc:648] HybridClock initialized: now 1786172779114828 us; error 39 us; skew 500 ppm
Aug 08 07:06:19 dist-test-slave-8tdl krb5kdc[3435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172779, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:19.122876 3448 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:06:19.123443 3448 webserver.cc:460] Webserver started at http://127.0.0.1:34261/ using document root <none> and password file <none>
I20260808 07:06:19.124078 3448 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:19.124195 3448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:19.124364 3448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:19.125528 3448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "260e3d7c7d9d48a689058ba43120b9d8"
format_stamp: "Formatted at 2026-08-08 07:06:19 on dist-test-slave-8tdl"
server_key: "eaaa4891e120349f0b01730fd0d2538c"
server_key_iv: "478355bcd73255cb66dbfb3aa9a35799"
server_key_version: "encryptionkey@0"
I20260808 07:06:19.125922 3448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "260e3d7c7d9d48a689058ba43120b9d8"
format_stamp: "Formatted at 2026-08-08 07:06:19 on dist-test-slave-8tdl"
server_key: "eaaa4891e120349f0b01730fd0d2538c"
server_key_iv: "478355bcd73255cb66dbfb3aa9a35799"
server_key_version: "encryptionkey@0"
I20260808 07:06:19.127650 3448 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 07:06:19.128719 3463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:19.129053 3448 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:19.129191 3448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "260e3d7c7d9d48a689058ba43120b9d8"
format_stamp: "Formatted at 2026-08-08 07:06:19 on dist-test-slave-8tdl"
server_key: "eaaa4891e120349f0b01730fd0d2538c"
server_key_iv: "478355bcd73255cb66dbfb3aa9a35799"
server_key_version: "encryptionkey@0"
I20260808 07:06:19.129288 3448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:19.134799 3448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:19.137915 3448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:19.138242 3448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:19.144378 3448 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37309
I20260808 07:06:19.144520 3519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37309 every 8 connection(s)
I20260808 07:06:19.144876 3448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:19.145969 3520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:19.148597 3520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 260e3d7c7d9d48a689058ba43120b9d8: Bootstrap starting.
I20260808 07:06:19.149214 3520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 260e3d7c7d9d48a689058ba43120b9d8: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:19.149498 3520 log.cc:824] T 00000000000000000000000000000000 P 260e3d7c7d9d48a689058ba43120b9d8: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:19.150262 3520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 260e3d7c7d9d48a689058ba43120b9d8: No bootstrap required, opened a new log
I20260808 07:06:19.151645 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 3448
I20260808 07:06:19.151780 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:19.151984 14140 external_mini_cluster.cc:1428] Setting key c08062bbcb0a1eb5212b5925faf879a6
I20260808 07:06:19.152962 3520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 260e3d7c7d9d48a689058ba43120b9d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "260e3d7c7d9d48a689058ba43120b9d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37309 } }
I20260808 07:06:19.153223 3520 raft_consensus.cc:374] T 00000000000000000000000000000000 P 260e3d7c7d9d48a689058ba43120b9d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:19.153301 3520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 260e3d7c7d9d48a689058ba43120b9d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 260e3d7c7d9d48a689058ba43120b9d8, State: Initialized, Role: FOLLOWER
I20260808 07:06:19.153463 3520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 260e3d7c7d9d48a689058ba43120b9d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "260e3d7c7d9d48a689058ba43120b9d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37309 } }
I20260808 07:06:19.153571 3520 raft_consensus.cc:388] T 00000000000000000000000000000000 P 260e3d7c7d9d48a689058ba43120b9d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:19.153633 3520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 260e3d7c7d9d48a689058ba43120b9d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:19.153692 3520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 260e3d7c7d9d48a689058ba43120b9d8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:19.154948 3520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 260e3d7c7d9d48a689058ba43120b9d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "260e3d7c7d9d48a689058ba43120b9d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37309 } }
I20260808 07:06:19.155102 3520 leader_election.cc:302] T 00000000000000000000000000000000 P 260e3d7c7d9d48a689058ba43120b9d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 260e3d7c7d9d48a689058ba43120b9d8; no voters:
I20260808 07:06:19.155335 3520 leader_election.cc:288] T 00000000000000000000000000000000 P 260e3d7c7d9d48a689058ba43120b9d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:19.155359 3525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 260e3d7c7d9d48a689058ba43120b9d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:19.155524 3525 raft_consensus.cc:686] T 00000000000000000000000000000000 P 260e3d7c7d9d48a689058ba43120b9d8 [term 1 LEADER]: Becoming Leader. State: Replica: 260e3d7c7d9d48a689058ba43120b9d8, State: Running, Role: LEADER
I20260808 07:06:19.155637 3525 consensus_queue.cc:237] T 00000000000000000000000000000000 P 260e3d7c7d9d48a689058ba43120b9d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "260e3d7c7d9d48a689058ba43120b9d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37309 } }
I20260808 07:06:19.155673 3520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 260e3d7c7d9d48a689058ba43120b9d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:19.156203 3526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 260e3d7c7d9d48a689058ba43120b9d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 260e3d7c7d9d48a689058ba43120b9d8. Latest consensus state: current_term: 1 leader_uuid: "260e3d7c7d9d48a689058ba43120b9d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "260e3d7c7d9d48a689058ba43120b9d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37309 } } }
I20260808 07:06:19.156127 3525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 260e3d7c7d9d48a689058ba43120b9d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "260e3d7c7d9d48a689058ba43120b9d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "260e3d7c7d9d48a689058ba43120b9d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37309 } } }
I20260808 07:06:19.156340 3525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 260e3d7c7d9d48a689058ba43120b9d8 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:19.156355 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:19.153007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48165 (local address 127.0.0.1:37309)
0808 07:06:19.153638 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:19.153650 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:19.153715 (+ 65us) server_negotiation.cc:407] Connection header received
0808 07:06:19.153853 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:19.153859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:19.153884 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:19.154044 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:19.154979 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:19.154985 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:19.154986 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:19.154997 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:19.155424 (+ 427us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:19.155508 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:19.155862 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:19.156000 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 07:06:19.156061 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":112,"threads_started":1}
I20260808 07:06:19.156808 3529 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:19.157363 3529 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:19.157500 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:19.157624 3526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 260e3d7c7d9d48a689058ba43120b9d8 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:19.159627 3529 catalog_manager.cc:1347] Generated new cluster ID: 894d3156085f4109a07a4790e3a3bb46
I20260808 07:06:19.159693 3529 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:19.160676 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:19.159359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46487 (local address 127.0.0.1:37309)
0808 07:06:19.159401 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:19.159404 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:19.159581 (+ 177us) server_negotiation.cc:407] Connection header received
0808 07:06:19.159677 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:19.159678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:19.159684 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:19.159739 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:19.160419 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:19.160420 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:19.160421 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:19.160425 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:19.160442 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:19.160447 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:19.160506 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:19.160572 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 07:06:19.160586 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:06:19.169548 3529 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:19.169811 3529 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:19.175932 3529 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 260e3d7c7d9d48a689058ba43120b9d8: Generated new TSK 0
I20260808 07:06:19.176172 3529 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:19.185611 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 3448
2026-08-08T07:06:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1460 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:19 dist-test-slave-8tdl krb5kdc[3552](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:19 dist-test-slave-8tdl krb5kdc[3552](info): set up 2 sockets
Aug 08 07:06:19 dist-test-slave-8tdl krb5kdc[3552](info): commencing operation
krb5kdc: starting...
W20260808 07:06:20.323793 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:06:20 dist-test-slave-8tdl krb5kdc[3552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:06:20.472263 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42551
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:20.560228 3568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:20.560659 3568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:20.560714 3568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:20.562119 3568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:20.562206 3568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:20.562233 3568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:20.562256 3568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:20.562286 3568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:20.564028 3568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:20.564369 3568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:20.564633 3568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:20.570489 3573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:20.570482 3576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:20.570509 3568 server_base.cc:1034] running on GCE node
W20260808 07:06:20.570488 3574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:20.571599 3568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:20.572086 3568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:20.573295 3568 hybrid_clock.cc:648] HybridClock initialized: now 1786172780573264 us; error 52 us; skew 500 ppm
Aug 08 07:06:20 dist-test-slave-8tdl krb5kdc[3552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:20.576912 3568 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:06:20.577653 3568 webserver.cc:460] Webserver started at http://127.0.0.1:45585/ using document root <none> and password file <none>
I20260808 07:06:20.577919 3568 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:20.577994 3568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:20.578161 3568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:20.579365 3568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "43c4cf153c014953814dd96a207e846f"
format_stamp: "Formatted at 2026-08-08 07:06:20 on dist-test-slave-8tdl"
server_key: "c99752379d3941658551451a2a71ba13"
server_key_iv: "60eaadd5ccfce4ba4461c6922b9fe7e0"
server_key_version: "encryptionkey@0"
I20260808 07:06:20.579757 3568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "43c4cf153c014953814dd96a207e846f"
format_stamp: "Formatted at 2026-08-08 07:06:20 on dist-test-slave-8tdl"
server_key: "c99752379d3941658551451a2a71ba13"
server_key_iv: "60eaadd5ccfce4ba4461c6922b9fe7e0"
server_key_version: "encryptionkey@0"
I20260808 07:06:20.581360 3568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:06:20.582376 3584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:20.582718 3568 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:20.582930 3568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "43c4cf153c014953814dd96a207e846f"
format_stamp: "Formatted at 2026-08-08 07:06:20 on dist-test-slave-8tdl"
server_key: "c99752379d3941658551451a2a71ba13"
server_key_iv: "60eaadd5ccfce4ba4461c6922b9fe7e0"
server_key_version: "encryptionkey@0"
I20260808 07:06:20.583029 3568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:20.611984 3568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:20.614794 3568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:20.615080 3568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:20.619766 3568 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42551
I20260808 07:06:20.619927 3637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42551 every 8 connection(s)
I20260808 07:06:20.620236 3568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:20.621243 3638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:20.623899 3638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43c4cf153c014953814dd96a207e846f: Bootstrap starting.
I20260808 07:06:20.624542 3638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43c4cf153c014953814dd96a207e846f: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:20.624835 3638 log.cc:824] T 00000000000000000000000000000000 P 43c4cf153c014953814dd96a207e846f: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:20.625741 3638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43c4cf153c014953814dd96a207e846f: No bootstrap required, opened a new log
I20260808 07:06:20.627805 3638 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43c4cf153c014953814dd96a207e846f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c4cf153c014953814dd96a207e846f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42551 } }
I20260808 07:06:20.627978 3638 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43c4cf153c014953814dd96a207e846f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:20.628005 3638 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43c4cf153c014953814dd96a207e846f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43c4cf153c014953814dd96a207e846f, State: Initialized, Role: FOLLOWER
I20260808 07:06:20.628165 3638 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43c4cf153c014953814dd96a207e846f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c4cf153c014953814dd96a207e846f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42551 } }
I20260808 07:06:20.628263 3638 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43c4cf153c014953814dd96a207e846f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:20.628289 3638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43c4cf153c014953814dd96a207e846f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:20.628320 3638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43c4cf153c014953814dd96a207e846f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:20.628971 3638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43c4cf153c014953814dd96a207e846f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c4cf153c014953814dd96a207e846f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42551 } }
I20260808 07:06:20.629065 3638 leader_election.cc:302] T 00000000000000000000000000000000 P 43c4cf153c014953814dd96a207e846f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43c4cf153c014953814dd96a207e846f; no voters:
I20260808 07:06:20.629238 3638 leader_election.cc:288] T 00000000000000000000000000000000 P 43c4cf153c014953814dd96a207e846f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:20.629308 3643 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43c4cf153c014953814dd96a207e846f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:20.629446 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 3568
I20260808 07:06:20.629527 3643 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43c4cf153c014953814dd96a207e846f [term 1 LEADER]: Becoming Leader. State: Replica: 43c4cf153c014953814dd96a207e846f, State: Running, Role: LEADER
I20260808 07:06:20.629558 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:20.629566 3638 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43c4cf153c014953814dd96a207e846f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:20.629621 3643 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43c4cf153c014953814dd96a207e846f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c4cf153c014953814dd96a207e846f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42551 } }
I20260808 07:06:20.629778 14140 external_mini_cluster.cc:1428] Setting key e3bd781db7136b4faf7b6f30005b9039
I20260808 07:06:20.630208 3644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43c4cf153c014953814dd96a207e846f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43c4cf153c014953814dd96a207e846f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c4cf153c014953814dd96a207e846f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42551 } } }
I20260808 07:06:20.630301 3644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43c4cf153c014953814dd96a207e846f [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:20.630759 3645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43c4cf153c014953814dd96a207e846f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43c4cf153c014953814dd96a207e846f. Latest consensus state: current_term: 1 leader_uuid: "43c4cf153c014953814dd96a207e846f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c4cf153c014953814dd96a207e846f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42551 } } }
I20260808 07:06:20.630946 3645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43c4cf153c014953814dd96a207e846f [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:20.631825 3654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:20.632416 3654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:20.635200 3654 catalog_manager.cc:1347] Generated new cluster ID: 777a1624a03246bba63ccdb9fe3e0435
I20260808 07:06:20.635257 3654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:06:20 dist-test-slave-8tdl krb5kdc[3552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 07:06:20.643224 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:20.630573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:32985 (local address 127.0.0.1:42551)
0808 07:06:20.630836 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:20.630846 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:20.630981 (+ 135us) server_negotiation.cc:407] Connection header received
0808 07:06:20.631107 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:20.631114 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:20.631173 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:20.631311 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:20.632263 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:20.633185 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:20.634028 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:20.634757 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:20.634891 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:20.638039 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:20.638044 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:20.638044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:20.638058 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:20.640848 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:20.641379 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:20.641383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:20.641383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:20.641511 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:20.641771 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:20.641772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:20.641772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:20.642115 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:20.642281 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:20.642833 (+ 552us) server_negotiation.cc:299] Negotiation successful
0808 07:06:20.642950 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":109,"threads_started":1}
I20260808 07:06:20.644328 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:20.648636 3654 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:20.649425 3654 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:20.653952 3654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43c4cf153c014953814dd96a207e846f: Generated new TSK 0
I20260808 07:06:20.654170 3654 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:20.654604 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:20.645971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:43073 (local address 127.0.0.1:42551)
0808 07:06:20.646008 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:20.646010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:20.646120 (+ 110us) server_negotiation.cc:407] Connection header received
0808 07:06:20.646174 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:20.646175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:20.646186 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:20.646248 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:20.646980 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:20.647725 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:20.649027 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:20.649337 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:20.649419 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:20.651045 (+ 1626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:20.651046 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:20.651047 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:20.651055 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:20.652875 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:20.653382 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:20.653383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:20.653384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:20.653453 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:20.653895 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:20.653896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:20.653897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:20.654049 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:20.654156 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:20.654435 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 07:06:20.654504 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:06:20.657243 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 3568
W20260808 07:06:20.663841 3550 connection.cc:535] client connection to 127.0.0.1:42551 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42551): Connection reset by peer (error 104)
2026-08-08T07:06:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1471 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:20 dist-test-slave-8tdl krb5kdc[3676](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:20 dist-test-slave-8tdl krb5kdc[3676](info): set up 2 sockets
Aug 08 07:06:20 dist-test-slave-8tdl krb5kdc[3676](info): commencing operation
krb5kdc: starting...
W20260808 07:06:21.812783 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:06:21 dist-test-slave-8tdl krb5kdc[3676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:06:21.958930 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38105
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:22.055426 3690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:22.055680 3690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:22.055709 3690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:22.057036 3690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:22.057094 3690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:22.057111 3690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:22.057123 3690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:22.057137 3690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:22.058830 3690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:22.059072 3690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:22.059270 3690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:06:22.064750 3690 server_base.cc:1034] running on GCE node
W20260808 07:06:22.064736 3696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:22.064761 3698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:22.064781 3695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:22.065289 3690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:22.065635 3690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:22.066891 3690 hybrid_clock.cc:648] HybridClock initialized: now 1786172782066747 us; error 56 us; skew 500 ppm
Aug 08 07:06:22 dist-test-slave-8tdl krb5kdc[3676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172782, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:22.069783 3690 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:06:22.070266 3690 webserver.cc:460] Webserver started at http://127.0.0.1:36649/ using document root <none> and password file <none>
I20260808 07:06:22.070497 3690 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:22.070552 3690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:22.070734 3690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:22.072237 3690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "30af723f5679484798e10aa43b22705d"
format_stamp: "Formatted at 2026-08-08 07:06:22 on dist-test-slave-8tdl"
server_key: "ac0450c8dfb39731ef88c501bbfec1cf"
server_key_iv: "e34cfa8e9e4d86a85bec32954c105437"
server_key_version: "encryptionkey@0"
I20260808 07:06:22.072758 3690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "30af723f5679484798e10aa43b22705d"
format_stamp: "Formatted at 2026-08-08 07:06:22 on dist-test-slave-8tdl"
server_key: "ac0450c8dfb39731ef88c501bbfec1cf"
server_key_iv: "e34cfa8e9e4d86a85bec32954c105437"
server_key_version: "encryptionkey@0"
I20260808 07:06:22.074784 3690 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 07:06:22.075826 3705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:22.076296 3690 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:22.076524 3690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "30af723f5679484798e10aa43b22705d"
format_stamp: "Formatted at 2026-08-08 07:06:22 on dist-test-slave-8tdl"
server_key: "ac0450c8dfb39731ef88c501bbfec1cf"
server_key_iv: "e34cfa8e9e4d86a85bec32954c105437"
server_key_version: "encryptionkey@0"
I20260808 07:06:22.076627 3690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:22.097915 3690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:22.101716 3690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:22.101855 3690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:22.107906 3690 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38105
I20260808 07:06:22.107982 3759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38105 every 8 connection(s)
I20260808 07:06:22.108378 3690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:22.109356 3760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:22.112284 3760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30af723f5679484798e10aa43b22705d: Bootstrap starting.
I20260808 07:06:22.112996 3760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30af723f5679484798e10aa43b22705d: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:22.113353 3760 log.cc:824] T 00000000000000000000000000000000 P 30af723f5679484798e10aa43b22705d: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:22.114439 3760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30af723f5679484798e10aa43b22705d: No bootstrap required, opened a new log
I20260808 07:06:22.116618 3760 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30af723f5679484798e10aa43b22705d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30af723f5679484798e10aa43b22705d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38105 } }
I20260808 07:06:22.116775 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 3690
I20260808 07:06:22.116847 3760 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30af723f5679484798e10aa43b22705d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:22.116899 3760 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30af723f5679484798e10aa43b22705d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30af723f5679484798e10aa43b22705d, State: Initialized, Role: FOLLOWER
I20260808 07:06:22.117108 3760 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30af723f5679484798e10aa43b22705d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30af723f5679484798e10aa43b22705d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38105 } }
I20260808 07:06:22.117223 3760 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30af723f5679484798e10aa43b22705d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:22.117103 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:22.117337 3760 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30af723f5679484798e10aa43b22705d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:22.117400 3760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30af723f5679484798e10aa43b22705d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:22.117648 14140 external_mini_cluster.cc:1428] Setting key 862e7ae2f599bd1bc5a2ef2b91d4ebe5
I20260808 07:06:22.118357 3760 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30af723f5679484798e10aa43b22705d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30af723f5679484798e10aa43b22705d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38105 } }
I20260808 07:06:22.118497 3760 leader_election.cc:302] T 00000000000000000000000000000000 P 30af723f5679484798e10aa43b22705d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30af723f5679484798e10aa43b22705d; no voters:
I20260808 07:06:22.118748 3760 leader_election.cc:288] T 00000000000000000000000000000000 P 30af723f5679484798e10aa43b22705d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:22.118767 3765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30af723f5679484798e10aa43b22705d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:22.119053 3765 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30af723f5679484798e10aa43b22705d [term 1 LEADER]: Becoming Leader. State: Replica: 30af723f5679484798e10aa43b22705d, State: Running, Role: LEADER
I20260808 07:06:22.119160 3765 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30af723f5679484798e10aa43b22705d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30af723f5679484798e10aa43b22705d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38105 } }
I20260808 07:06:22.119230 3760 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30af723f5679484798e10aa43b22705d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:22.119675 3765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30af723f5679484798e10aa43b22705d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30af723f5679484798e10aa43b22705d. Latest consensus state: current_term: 1 leader_uuid: "30af723f5679484798e10aa43b22705d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30af723f5679484798e10aa43b22705d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38105 } } }
I20260808 07:06:22.119794 3765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30af723f5679484798e10aa43b22705d [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:22.120029 3771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:22.120311 3766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30af723f5679484798e10aa43b22705d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30af723f5679484798e10aa43b22705d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30af723f5679484798e10aa43b22705d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38105 } } }
I20260808 07:06:22.120870 3766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30af723f5679484798e10aa43b22705d [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:22.121268 3771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:22.123275 3771 catalog_manager.cc:1347] Generated new cluster ID: e200a806510346fbbe071bf88de94341
I20260808 07:06:22.123344 3771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:22.126391 3764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:22.118403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:55487 (local address 127.0.0.1:38105)
0808 07:06:22.118654 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:22.118664 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:22.118739 (+ 75us) server_negotiation.cc:407] Connection header received
0808 07:06:22.119183 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:22.119194 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:22.119382 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:22.120128 (+ 746us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:22.120268 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:22.123410 (+ 3142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:22.124290 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:22.124730 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:22.124858 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:22.125711 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:22.125716 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:22.125717 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:22.125920 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:22.125953 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:22.125964 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:22.126072 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:22.126155 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 07:06:22.126188 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":128,"threads_started":1}
I20260808 07:06:22.127679 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:22.135998 3764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:22.131161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:41421 (local address 127.0.0.1:38105)
0808 07:06:22.131240 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:22.131242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:22.131516 (+ 274us) server_negotiation.cc:407] Connection header received
0808 07:06:22.131619 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:22.131621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:22.131635 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:22.131697 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:22.131875 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:22.132629 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:22.133915 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:22.134351 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:22.134537 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:22.135514 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:22.135516 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:22.135516 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:22.135522 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:22.135539 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:22.135547 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:22.135624 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:22.135699 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 07:06:22.135715 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 07:06:22.144637 3771 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:22.145421 3771 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:22.156790 3771 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30af723f5679484798e10aa43b22705d: Generated new TSK 0
I20260808 07:06:22.157068 3771 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:22.163836 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 3690
W20260808 07:06:22.169826 3674 connection.cc:535] client connection to 127.0.0.1:38105 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38105): Connection reset by peer (error 104)
2026-08-08T07:06:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1506 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:22 dist-test-slave-8tdl krb5kdc[3792](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:22 dist-test-slave-8tdl krb5kdc[3792](info): set up 2 sockets
Aug 08 07:06:22 dist-test-slave-8tdl krb5kdc[3792](info): commencing operation
krb5kdc: starting...
W20260808 07:06:23.270411 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:06:23 dist-test-slave-8tdl krb5kdc[3792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:06:23.415755 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46629
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:23.502978 3806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:23.503294 3806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:23.503343 3806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:23.504640 3806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:23.504724 3806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:23.504747 3806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:23.504771 3806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:23.504797 3806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:23.506422 3806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:23.506790 3806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:23.507048 3806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:23.512507 3813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:23.512432 3812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:23.512406 3806 server_base.cc:1034] running on GCE node
W20260808 07:06:23.512403 3815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:23.513005 3806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:23.513350 3806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:23.514628 3806 hybrid_clock.cc:648] HybridClock initialized: now 1786172783514569 us; error 84 us; skew 500 ppm
Aug 08 07:06:23 dist-test-slave-8tdl krb5kdc[3792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172783, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:23.517786 3806 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:06:23.518285 3806 webserver.cc:460] Webserver started at http://127.0.0.1:38303/ using document root <none> and password file <none>
I20260808 07:06:23.518579 3806 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:23.518635 3806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:23.518847 3806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:23.520151 3806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "04eb3747d86b44ca9f2d7749a082bdc9"
format_stamp: "Formatted at 2026-08-08 07:06:23 on dist-test-slave-8tdl"
server_key: "c3764ceddf3a5c800b968c09d06cf0c9"
server_key_iv: "cc057ec73d1d4f69f420fc5a24f08377"
server_key_version: "encryptionkey@0"
I20260808 07:06:23.520804 3806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "04eb3747d86b44ca9f2d7749a082bdc9"
format_stamp: "Formatted at 2026-08-08 07:06:23 on dist-test-slave-8tdl"
server_key: "c3764ceddf3a5c800b968c09d06cf0c9"
server_key_iv: "cc057ec73d1d4f69f420fc5a24f08377"
server_key_version: "encryptionkey@0"
I20260808 07:06:23.522490 3806 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:06:23.523613 3822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:23.523885 3806 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:23.524235 3806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "04eb3747d86b44ca9f2d7749a082bdc9"
format_stamp: "Formatted at 2026-08-08 07:06:23 on dist-test-slave-8tdl"
server_key: "c3764ceddf3a5c800b968c09d06cf0c9"
server_key_iv: "cc057ec73d1d4f69f420fc5a24f08377"
server_key_version: "encryptionkey@0"
I20260808 07:06:23.524351 3806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:23.528478 3806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:23.530781 3806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:23.531032 3806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:23.536051 3806 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46629
I20260808 07:06:23.536108 3878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46629 every 8 connection(s)
I20260808 07:06:23.536533 3806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:23.537649 3879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:23.540316 3879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04eb3747d86b44ca9f2d7749a082bdc9: Bootstrap starting.
I20260808 07:06:23.540912 3879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04eb3747d86b44ca9f2d7749a082bdc9: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:23.541177 3879 log.cc:824] T 00000000000000000000000000000000 P 04eb3747d86b44ca9f2d7749a082bdc9: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:23.541934 3879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04eb3747d86b44ca9f2d7749a082bdc9: No bootstrap required, opened a new log
I20260808 07:06:23.542523 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 3806
I20260808 07:06:23.542663 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:23.542906 14140 external_mini_cluster.cc:1428] Setting key e95c66c7f51076aa21bca623fa46dae3
I20260808 07:06:23.544924 3879 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04eb3747d86b44ca9f2d7749a082bdc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04eb3747d86b44ca9f2d7749a082bdc9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46629 } }
I20260808 07:06:23.545158 3879 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04eb3747d86b44ca9f2d7749a082bdc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:23.545226 3879 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04eb3747d86b44ca9f2d7749a082bdc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04eb3747d86b44ca9f2d7749a082bdc9, State: Initialized, Role: FOLLOWER
I20260808 07:06:23.545691 3879 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04eb3747d86b44ca9f2d7749a082bdc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04eb3747d86b44ca9f2d7749a082bdc9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46629 } }
I20260808 07:06:23.545863 3879 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04eb3747d86b44ca9f2d7749a082bdc9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:23.545928 3879 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04eb3747d86b44ca9f2d7749a082bdc9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:23.546036 3879 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04eb3747d86b44ca9f2d7749a082bdc9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:23.546779 3883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:23.543619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:57517 (local address 127.0.0.1:46629)
0808 07:06:23.544021 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:23.544031 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:23.544066 (+ 35us) server_negotiation.cc:407] Connection header received
0808 07:06:23.544872 (+ 806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:23.544882 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:23.544922 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:23.545093 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:23.545814 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:23.545819 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:23.545819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:23.545824 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:23.545877 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:23.545894 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:23.546153 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:23.546188 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 07:06:23.546235 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":119,"threads_started":1}
I20260808 07:06:23.547070 3879 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04eb3747d86b44ca9f2d7749a082bdc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04eb3747d86b44ca9f2d7749a082bdc9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46629 } }
I20260808 07:06:23.547210 3879 leader_election.cc:302] T 00000000000000000000000000000000 P 04eb3747d86b44ca9f2d7749a082bdc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04eb3747d86b44ca9f2d7749a082bdc9; no voters:
I20260808 07:06:23.547413 3879 leader_election.cc:288] T 00000000000000000000000000000000 P 04eb3747d86b44ca9f2d7749a082bdc9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:23.547482 3884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04eb3747d86b44ca9f2d7749a082bdc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:23.547631 3884 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04eb3747d86b44ca9f2d7749a082bdc9 [term 1 LEADER]: Becoming Leader. State: Replica: 04eb3747d86b44ca9f2d7749a082bdc9, State: Running, Role: LEADER
I20260808 07:06:23.547739 3879 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04eb3747d86b44ca9f2d7749a082bdc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:23.547806 3884 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04eb3747d86b44ca9f2d7749a082bdc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04eb3747d86b44ca9f2d7749a082bdc9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46629 } }
I20260808 07:06:23.548022 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:23.548173 3884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04eb3747d86b44ca9f2d7749a082bdc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04eb3747d86b44ca9f2d7749a082bdc9. Latest consensus state: current_term: 1 leader_uuid: "04eb3747d86b44ca9f2d7749a082bdc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04eb3747d86b44ca9f2d7749a082bdc9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46629 } } }
I20260808 07:06:23.548337 3884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04eb3747d86b44ca9f2d7749a082bdc9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:23.548614 3893 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:23.549697 3886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04eb3747d86b44ca9f2d7749a082bdc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04eb3747d86b44ca9f2d7749a082bdc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04eb3747d86b44ca9f2d7749a082bdc9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46629 } } }
I20260808 07:06:23.549786 3886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04eb3747d86b44ca9f2d7749a082bdc9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:23.550997 3893 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:23.559685 3893 catalog_manager.cc:1347] Generated new cluster ID: 6c5729e6b0d94e8fae41d05bd9081755
I20260808 07:06:23.559783 3893 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:23.560145 3883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:23.551224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:40177 (local address 127.0.0.1:46629)
0808 07:06:23.551277 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:23.551279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:23.558894 (+ 7615us) server_negotiation.cc:407] Connection header received
0808 07:06:23.558938 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:23.558939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:23.558946 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:23.559034 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:23.559841 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:23.559842 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:23.559843 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:23.559849 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:23.559866 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:23.559872 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:23.559908 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:23.560048 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 07:06:23.560068 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 07:06:23.575795 3893 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:23.576045 3893 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:23.584690 3893 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04eb3747d86b44ca9f2d7749a082bdc9: Generated new TSK 0
I20260808 07:06:23.584952 3893 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:23.587666 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 3806
2026-08-08T07:06:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1425 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:23 dist-test-slave-8tdl krb5kdc[3910](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:23 dist-test-slave-8tdl krb5kdc[3910](info): set up 2 sockets
Aug 08 07:06:23 dist-test-slave-8tdl krb5kdc[3910](info): commencing operation
krb5kdc: starting...
W20260808 07:06:24.699021 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:06:24 dist-test-slave-8tdl krb5kdc[3910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:06:24.843187 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43953
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:24.931903 3923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:24.932152 3923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:24.932179 3923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:24.933490 3923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:24.933553 3923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:24.933569 3923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:24.933583 3923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:24.933594 3923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:24.935302 3923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43953
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:24.935616 3923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:24.935881 3923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:24.941774 3930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:24.941800 3932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:24.941774 3929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:24.942447 3923 server_base.cc:1034] running on GCE node
I20260808 07:06:24.942795 3923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:24.943177 3923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:24.944386 3923 hybrid_clock.cc:648] HybridClock initialized: now 1786172784944356 us; error 54 us; skew 500 ppm
Aug 08 07:06:24 dist-test-slave-8tdl krb5kdc[3910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172784, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:24.957218 3923 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:06:24.957998 3923 webserver.cc:460] Webserver started at http://127.0.0.1:43367/ using document root <none> and password file <none>
I20260808 07:06:24.958292 3923 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:24.958349 3923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:24.958606 3923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:24.959856 3923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "326f878b845b4c1ca4c98e4211410d1e"
format_stamp: "Formatted at 2026-08-08 07:06:24 on dist-test-slave-8tdl"
server_key: "9cc0e41b64e09e2f352f02dcfbe832fb"
server_key_iv: "78016ef49aaab61e1faa14f5029e28c1"
server_key_version: "encryptionkey@0"
I20260808 07:06:24.960294 3923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "326f878b845b4c1ca4c98e4211410d1e"
format_stamp: "Formatted at 2026-08-08 07:06:24 on dist-test-slave-8tdl"
server_key: "9cc0e41b64e09e2f352f02dcfbe832fb"
server_key_iv: "78016ef49aaab61e1faa14f5029e28c1"
server_key_version: "encryptionkey@0"
I20260808 07:06:24.961937 3923 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:06:24.962929 3939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:24.963224 3923 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:24.963356 3923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "326f878b845b4c1ca4c98e4211410d1e"
format_stamp: "Formatted at 2026-08-08 07:06:24 on dist-test-slave-8tdl"
server_key: "9cc0e41b64e09e2f352f02dcfbe832fb"
server_key_iv: "78016ef49aaab61e1faa14f5029e28c1"
server_key_version: "encryptionkey@0"
I20260808 07:06:24.963474 3923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:24.992870 3923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:24.996016 3923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:24.996223 3923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:25.000989 3923 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43953
I20260808 07:06:25.001169 3998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43953 every 8 connection(s)
I20260808 07:06:25.001490 3923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:25.002580 3999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:25.005457 3999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 326f878b845b4c1ca4c98e4211410d1e: Bootstrap starting.
I20260808 07:06:25.006206 3999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 326f878b845b4c1ca4c98e4211410d1e: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:25.006585 3999 log.cc:824] T 00000000000000000000000000000000 P 326f878b845b4c1ca4c98e4211410d1e: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:25.007659 3999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 326f878b845b4c1ca4c98e4211410d1e: No bootstrap required, opened a new log
I20260808 07:06:25.010129 3999 raft_consensus.cc:348] T 00000000000000000000000000000000 P 326f878b845b4c1ca4c98e4211410d1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326f878b845b4c1ca4c98e4211410d1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43953 } }
I20260808 07:06:25.010331 3999 raft_consensus.cc:374] T 00000000000000000000000000000000 P 326f878b845b4c1ca4c98e4211410d1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:25.010358 3999 raft_consensus.cc:729] T 00000000000000000000000000000000 P 326f878b845b4c1ca4c98e4211410d1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 326f878b845b4c1ca4c98e4211410d1e, State: Initialized, Role: FOLLOWER
I20260808 07:06:25.010576 3999 consensus_queue.cc:260] T 00000000000000000000000000000000 P 326f878b845b4c1ca4c98e4211410d1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326f878b845b4c1ca4c98e4211410d1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43953 } }
I20260808 07:06:25.010643 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 3923
I20260808 07:06:25.010721 3999 raft_consensus.cc:388] T 00000000000000000000000000000000 P 326f878b845b4c1ca4c98e4211410d1e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:25.010772 3999 raft_consensus.cc:482] T 00000000000000000000000000000000 P 326f878b845b4c1ca4c98e4211410d1e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:25.010803 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:25.010836 3999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 326f878b845b4c1ca4c98e4211410d1e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:25.011121 14140 external_mini_cluster.cc:1428] Setting key b6eace314ecab4051f0528f6d1c218d1
I20260808 07:06:25.011781 3999 raft_consensus.cc:504] T 00000000000000000000000000000000 P 326f878b845b4c1ca4c98e4211410d1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326f878b845b4c1ca4c98e4211410d1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43953 } }
I20260808 07:06:25.011919 3999 leader_election.cc:302] T 00000000000000000000000000000000 P 326f878b845b4c1ca4c98e4211410d1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 326f878b845b4c1ca4c98e4211410d1e; no voters:
I20260808 07:06:25.012094 3999 leader_election.cc:288] T 00000000000000000000000000000000 P 326f878b845b4c1ca4c98e4211410d1e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:25.012152 4004 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 326f878b845b4c1ca4c98e4211410d1e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:25.012404 4004 raft_consensus.cc:686] T 00000000000000000000000000000000 P 326f878b845b4c1ca4c98e4211410d1e [term 1 LEADER]: Becoming Leader. State: Replica: 326f878b845b4c1ca4c98e4211410d1e, State: Running, Role: LEADER
I20260808 07:06:25.012535 4004 consensus_queue.cc:237] T 00000000000000000000000000000000 P 326f878b845b4c1ca4c98e4211410d1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326f878b845b4c1ca4c98e4211410d1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43953 } }
I20260808 07:06:25.012657 3999 sys_catalog.cc:564] T 00000000000000000000000000000000 P 326f878b845b4c1ca4c98e4211410d1e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:25.013185 4007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 326f878b845b4c1ca4c98e4211410d1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 326f878b845b4c1ca4c98e4211410d1e. Latest consensus state: current_term: 1 leader_uuid: "326f878b845b4c1ca4c98e4211410d1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326f878b845b4c1ca4c98e4211410d1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43953 } } }
I20260808 07:06:25.013473 4007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 326f878b845b4c1ca4c98e4211410d1e [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:25.013387 4005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 326f878b845b4c1ca4c98e4211410d1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "326f878b845b4c1ca4c98e4211410d1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326f878b845b4c1ca4c98e4211410d1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43953 } } }
I20260808 07:06:25.013794 4005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 326f878b845b4c1ca4c98e4211410d1e [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:25.013901 4016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:25.014706 4016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:25.017028 4016 catalog_manager.cc:1347] Generated new cluster ID: 2d3979830e09434bb818579731e729ad
I20260808 07:06:25.017174 4016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:06:25 dist-test-slave-8tdl krb5kdc[3910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 07:06:25.026273 4003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.011746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:51607 (local address 127.0.0.1:43953)
0808 07:06:25.012024 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:25.012036 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:25.012106 (+ 70us) server_negotiation.cc:407] Connection header received
0808 07:06:25.012252 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:25.012261 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:25.012351 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:25.013481 (+ 1130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:25.013626 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.015400 (+ 1774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.016703 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.017605 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.017742 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.021386 (+ 3644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:25.021393 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:25.021393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:25.021410 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:25.024031 (+ 2621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:25.024764 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:25.024768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:25.024769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:25.024918 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:25.025190 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:25.025191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:25.025192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:25.025574 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:25.025775 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:25.025893 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 07:06:25.026010 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":107,"threads_started":1}
I20260808 07:06:25.027457 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:25.030622 4016 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:25.031781 4016 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:25.037896 4003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.029183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:40149 (local address 127.0.0.1:43953)
0808 07:06:25.029254 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:25.029256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:25.029527 (+ 271us) server_negotiation.cc:407] Connection header received
0808 07:06:25.029589 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:25.029590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:25.029631 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:25.029713 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:25.030402 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.031319 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.032541 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.032862 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.032933 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.034589 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:25.034590 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:25.034591 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:25.034598 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:25.036513 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:25.037002 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:25.037003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:25.037003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:25.037068 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:25.037314 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:25.037314 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:25.037315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:25.037447 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:25.037566 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:25.037724 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 07:06:25.037790 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:06:25.059429 4016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 326f878b845b4c1ca4c98e4211410d1e: Generated new TSK 0
I20260808 07:06:25.059733 4016 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:25.071386 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 3923
W20260808 07:06:25.078264 3908 connection.cc:535] client connection to 127.0.0.1:43953 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43953): Connection reset by peer (error 104)
2026-08-08T07:06:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1482 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:25 dist-test-slave-8tdl krb5kdc[4033](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:25 dist-test-slave-8tdl krb5kdc[4033](info): set up 2 sockets
Aug 08 07:06:25 dist-test-slave-8tdl krb5kdc[4033](info): commencing operation
krb5kdc: starting...
W20260808 07:06:26.230877 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:06:26 dist-test-slave-8tdl krb5kdc[4033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:06:26.370527 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36917
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:26.461959 4046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:26.462210 4046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:26.462236 4046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:26.463554 4046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:26.463613 4046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:26.463629 4046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:26.463642 4046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:26.463654 4046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:26.465205 4046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36917
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:26.465427 4046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:26.465620 4046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:26.471131 4051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:26.471261 4052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:26.471275 4054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:26.471920 4046 server_base.cc:1034] running on GCE node
I20260808 07:06:26.472218 4046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:26.472568 4046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:26.473769 4046 hybrid_clock.cc:648] HybridClock initialized: now 1786172786473742 us; error 57 us; skew 500 ppm
Aug 08 07:06:26 dist-test-slave-8tdl krb5kdc[4033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172786, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:26.476862 4046 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:06:26.477402 4046 webserver.cc:460] Webserver started at http://127.0.0.1:33557/ using document root <none> and password file <none>
I20260808 07:06:26.477660 4046 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:26.477713 4046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:26.477871 4046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:26.479221 4046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "57b5110cbf64407d8b9e95603f16eef9"
format_stamp: "Formatted at 2026-08-08 07:06:26 on dist-test-slave-8tdl"
server_key: "3b2d203fe8696bb1d9fcd5a08eb5dd82"
server_key_iv: "5ece3800c76ac0e20f9004ee5862d9af"
server_key_version: "encryptionkey@0"
I20260808 07:06:26.479621 4046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "57b5110cbf64407d8b9e95603f16eef9"
format_stamp: "Formatted at 2026-08-08 07:06:26 on dist-test-slave-8tdl"
server_key: "3b2d203fe8696bb1d9fcd5a08eb5dd82"
server_key_iv: "5ece3800c76ac0e20f9004ee5862d9af"
server_key_version: "encryptionkey@0"
I20260808 07:06:26.481261 4046 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:06:26.482227 4061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:26.482497 4046 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:26.482630 4046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "57b5110cbf64407d8b9e95603f16eef9"
format_stamp: "Formatted at 2026-08-08 07:06:26 on dist-test-slave-8tdl"
server_key: "3b2d203fe8696bb1d9fcd5a08eb5dd82"
server_key_iv: "5ece3800c76ac0e20f9004ee5862d9af"
server_key_version: "encryptionkey@0"
I20260808 07:06:26.482756 4046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:26.492950 4046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:26.495496 4046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:26.495682 4046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:26.500180 4046 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36917
I20260808 07:06:26.500612 4046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:26.500607 4117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36917 every 8 connection(s)
I20260808 07:06:26.501405 4118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:26.503926 4118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57b5110cbf64407d8b9e95603f16eef9: Bootstrap starting.
I20260808 07:06:26.504575 4118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57b5110cbf64407d8b9e95603f16eef9: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:26.504882 4118 log.cc:824] T 00000000000000000000000000000000 P 57b5110cbf64407d8b9e95603f16eef9: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:26.505774 4118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57b5110cbf64407d8b9e95603f16eef9: No bootstrap required, opened a new log
I20260808 07:06:26.507017 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 4046
I20260808 07:06:26.507153 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:26.507357 14140 external_mini_cluster.cc:1428] Setting key 11070a15c243419bf3d6ff8aa49ff7a8
I20260808 07:06:26.508055 4118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57b5110cbf64407d8b9e95603f16eef9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b5110cbf64407d8b9e95603f16eef9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36917 } }
I20260808 07:06:26.508237 4118 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57b5110cbf64407d8b9e95603f16eef9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:26.508277 4118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57b5110cbf64407d8b9e95603f16eef9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57b5110cbf64407d8b9e95603f16eef9, State: Initialized, Role: FOLLOWER
I20260808 07:06:26.508407 4118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57b5110cbf64407d8b9e95603f16eef9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b5110cbf64407d8b9e95603f16eef9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36917 } }
I20260808 07:06:26.508491 4118 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57b5110cbf64407d8b9e95603f16eef9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:26.508524 4118 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57b5110cbf64407d8b9e95603f16eef9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:26.508617 4118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57b5110cbf64407d8b9e95603f16eef9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:26.509955 4118 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57b5110cbf64407d8b9e95603f16eef9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b5110cbf64407d8b9e95603f16eef9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36917 } }
I20260808 07:06:26.510076 4118 leader_election.cc:302] T 00000000000000000000000000000000 P 57b5110cbf64407d8b9e95603f16eef9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57b5110cbf64407d8b9e95603f16eef9; no voters:
I20260808 07:06:26.510226 4118 leader_election.cc:288] T 00000000000000000000000000000000 P 57b5110cbf64407d8b9e95603f16eef9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:26.510308 4123 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57b5110cbf64407d8b9e95603f16eef9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:26.510483 4118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57b5110cbf64407d8b9e95603f16eef9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:26.510495 4123 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57b5110cbf64407d8b9e95603f16eef9 [term 1 LEADER]: Becoming Leader. State: Replica: 57b5110cbf64407d8b9e95603f16eef9, State: Running, Role: LEADER
I20260808 07:06:26.510632 4123 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57b5110cbf64407d8b9e95603f16eef9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b5110cbf64407d8b9e95603f16eef9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36917 } }
I20260808 07:06:26.511143 4124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57b5110cbf64407d8b9e95603f16eef9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57b5110cbf64407d8b9e95603f16eef9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b5110cbf64407d8b9e95603f16eef9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36917 } } }
I20260808 07:06:26.511444 4124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57b5110cbf64407d8b9e95603f16eef9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:26.511126 4125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57b5110cbf64407d8b9e95603f16eef9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57b5110cbf64407d8b9e95603f16eef9. Latest consensus state: current_term: 1 leader_uuid: "57b5110cbf64407d8b9e95603f16eef9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b5110cbf64407d8b9e95603f16eef9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36917 } } }
I20260808 07:06:26.511641 4125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57b5110cbf64407d8b9e95603f16eef9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:26.511672 4132 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:26.512275 4132 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:26.514302 4122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:26.508042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:54703 (local address 127.0.0.1:36917)
0808 07:06:26.508297 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:26.508307 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:26.508351 (+ 44us) server_negotiation.cc:407] Connection header received
0808 07:06:26.508469 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:26.508476 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:26.508541 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:26.508699 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:26.508938 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.510437 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.511748 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.512439 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.512529 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:26.513542 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:26.513545 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:26.513546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:26.513758 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:26.513799 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:26.513815 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:26.513904 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:26.514054 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 07:06:26.514084 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":129,"threads_started":1}
I20260808 07:06:26.514389 4132 catalog_manager.cc:1347] Generated new cluster ID: a9c2ffb7269a499f83eb50460422a4c3
I20260808 07:06:26.514442 4132 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:26.515539 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:26.522909 4132 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:26.523659 4132 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:26.525707 4122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:26.519143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:34055 (local address 127.0.0.1:36917)
0808 07:06:26.519191 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:26.519193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:26.522418 (+ 3225us) server_negotiation.cc:407] Connection header received
0808 07:06:26.522708 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:26.522710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:26.522723 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:26.522785 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:26.523062 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.523546 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.524347 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.524569 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.524660 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:26.525421 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:26.525423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:26.525423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:26.525428 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:26.525445 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:26.525449 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:26.525522 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:26.525589 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 07:06:26.525604 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:06:26.530782 4132 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57b5110cbf64407d8b9e95603f16eef9: Generated new TSK 0
I20260808 07:06:26.530995 4132 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:26.538951 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 4046
W20260808 07:06:26.544773 4031 connection.cc:535] client connection to 127.0.0.1:36917 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36917): Connection reset by peer (error 104)
2026-08-08T07:06:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1466 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:26 dist-test-slave-8tdl krb5kdc[4149](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:26 dist-test-slave-8tdl krb5kdc[4149](info): set up 2 sockets
Aug 08 07:06:26 dist-test-slave-8tdl krb5kdc[4149](info): commencing operation
krb5kdc: starting...
W20260808 07:06:27.650589 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:06:27 dist-test-slave-8tdl krb5kdc[4149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:06:27.800719 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38487
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:27.889298 4164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:27.889568 4164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:27.889616 4164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:27.890946 4164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:27.891029 4164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:27.891057 4164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:27.891088 4164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:27.891119 4164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:27.892885 4164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38487
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:27.893229 4164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:27.893499 4164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:27.899055 4173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:27.899122 4170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:27.899055 4171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:27.899281 4164 server_base.cc:1034] running on GCE node
I20260808 07:06:27.899585 4164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:27.899896 4164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:27.901072 4164 hybrid_clock.cc:648] HybridClock initialized: now 1786172787901061 us; error 35 us; skew 500 ppm
Aug 08 07:06:27 dist-test-slave-8tdl krb5kdc[4149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172787, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:27.903985 4164 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:06:27.904626 4164 webserver.cc:460] Webserver started at http://127.0.0.1:43395/ using document root <none> and password file <none>
I20260808 07:06:27.904870 4164 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:27.904917 4164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:27.905018 4164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:27.906193 4164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "fe2daf2e23a044bfaf5c3ea9e96f71a9"
format_stamp: "Formatted at 2026-08-08 07:06:27 on dist-test-slave-8tdl"
server_key: "b3a495d4d16cdcd2fa76368dbe5f6042"
server_key_iv: "0f6367d5f1cb72f60c90a0918782628c"
server_key_version: "encryptionkey@0"
I20260808 07:06:27.906620 4164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "fe2daf2e23a044bfaf5c3ea9e96f71a9"
format_stamp: "Formatted at 2026-08-08 07:06:27 on dist-test-slave-8tdl"
server_key: "b3a495d4d16cdcd2fa76368dbe5f6042"
server_key_iv: "0f6367d5f1cb72f60c90a0918782628c"
server_key_version: "encryptionkey@0"
I20260808 07:06:27.908268 4164 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:27.909188 4180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:27.909539 4164 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:27.909637 4164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "fe2daf2e23a044bfaf5c3ea9e96f71a9"
format_stamp: "Formatted at 2026-08-08 07:06:27 on dist-test-slave-8tdl"
server_key: "b3a495d4d16cdcd2fa76368dbe5f6042"
server_key_iv: "0f6367d5f1cb72f60c90a0918782628c"
server_key_version: "encryptionkey@0"
I20260808 07:06:27.909705 4164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:27.913555 4164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:27.916198 4164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:27.916333 4164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:27.921084 4164 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38487
I20260808 07:06:27.921114 4235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38487 every 8 connection(s)
I20260808 07:06:27.921489 4164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:27.922446 4236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:27.925045 4236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe2daf2e23a044bfaf5c3ea9e96f71a9: Bootstrap starting.
I20260808 07:06:27.925730 4236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe2daf2e23a044bfaf5c3ea9e96f71a9: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:27.926074 4236 log.cc:824] T 00000000000000000000000000000000 P fe2daf2e23a044bfaf5c3ea9e96f71a9: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:27.927045 4236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe2daf2e23a044bfaf5c3ea9e96f71a9: No bootstrap required, opened a new log
I20260808 07:06:27.927610 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 4164
I20260808 07:06:27.927731 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:27.927937 14140 external_mini_cluster.cc:1428] Setting key 998ebffefb46f6f8d05c1ca794754a68
I20260808 07:06:27.929395 4236 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe2daf2e23a044bfaf5c3ea9e96f71a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe2daf2e23a044bfaf5c3ea9e96f71a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38487 } }
I20260808 07:06:27.929553 4236 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe2daf2e23a044bfaf5c3ea9e96f71a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:27.929577 4236 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe2daf2e23a044bfaf5c3ea9e96f71a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe2daf2e23a044bfaf5c3ea9e96f71a9, State: Initialized, Role: FOLLOWER
I20260808 07:06:27.929726 4236 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe2daf2e23a044bfaf5c3ea9e96f71a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe2daf2e23a044bfaf5c3ea9e96f71a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38487 } }
I20260808 07:06:27.929827 4236 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe2daf2e23a044bfaf5c3ea9e96f71a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:27.929893 4236 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe2daf2e23a044bfaf5c3ea9e96f71a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:27.929989 4236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe2daf2e23a044bfaf5c3ea9e96f71a9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:27.931011 4240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:27.928522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:38199 (local address 127.0.0.1:38487)
0808 07:06:27.928747 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:27.928758 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:27.928897 (+ 139us) server_negotiation.cc:407] Connection header received
0808 07:06:27.929421 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:27.929429 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:27.929462 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:27.929647 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:27.930490 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:27.930494 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:27.930494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:27.930499 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:27.930556 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:27.930569 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:27.930640 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:27.930763 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 07:06:27.930813 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":95,"threads_started":1}
I20260808 07:06:27.931425 4236 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe2daf2e23a044bfaf5c3ea9e96f71a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe2daf2e23a044bfaf5c3ea9e96f71a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38487 } }
I20260808 07:06:27.931561 4236 leader_election.cc:302] T 00000000000000000000000000000000 P fe2daf2e23a044bfaf5c3ea9e96f71a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe2daf2e23a044bfaf5c3ea9e96f71a9; no voters:
I20260808 07:06:27.931740 4236 leader_election.cc:288] T 00000000000000000000000000000000 P fe2daf2e23a044bfaf5c3ea9e96f71a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:27.931800 4241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe2daf2e23a044bfaf5c3ea9e96f71a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:27.932013 4236 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe2daf2e23a044bfaf5c3ea9e96f71a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:27.932093 4241 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe2daf2e23a044bfaf5c3ea9e96f71a9 [term 1 LEADER]: Becoming Leader. State: Replica: fe2daf2e23a044bfaf5c3ea9e96f71a9, State: Running, Role: LEADER
I20260808 07:06:27.932126 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:27.932216 4241 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe2daf2e23a044bfaf5c3ea9e96f71a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe2daf2e23a044bfaf5c3ea9e96f71a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38487 } }
I20260808 07:06:27.932613 4241 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe2daf2e23a044bfaf5c3ea9e96f71a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe2daf2e23a044bfaf5c3ea9e96f71a9. Latest consensus state: current_term: 1 leader_uuid: "fe2daf2e23a044bfaf5c3ea9e96f71a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe2daf2e23a044bfaf5c3ea9e96f71a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38487 } } }
I20260808 07:06:27.932716 4241 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe2daf2e23a044bfaf5c3ea9e96f71a9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:27.932946 4250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:27.933529 4250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:27.933758 4242 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe2daf2e23a044bfaf5c3ea9e96f71a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe2daf2e23a044bfaf5c3ea9e96f71a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe2daf2e23a044bfaf5c3ea9e96f71a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38487 } } }
I20260808 07:06:27.934115 4242 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe2daf2e23a044bfaf5c3ea9e96f71a9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:27.935855 4240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:27.933921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:45867 (local address 127.0.0.1:38487)
0808 07:06:27.933971 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:27.933973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:27.934252 (+ 279us) server_negotiation.cc:407] Connection header received
0808 07:06:27.934847 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:27.934848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:27.934854 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:27.934914 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:27.935562 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:27.935564 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:27.935564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:27.935569 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:27.935587 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:27.935592 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:27.935644 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:27.935784 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 07:06:27.935798 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:06:27.936004 4250 catalog_manager.cc:1347] Generated new cluster ID: 0287d0a458eb49aca34113f73d5dbafc
I20260808 07:06:27.936084 4250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:27.947574 4250 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:27.947780 4250 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:27.952495 4250 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe2daf2e23a044bfaf5c3ea9e96f71a9: Generated new TSK 0
I20260808 07:06:27.952739 4250 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:27.956605 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 4164
2026-08-08T07:06:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1418 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:28 dist-test-slave-8tdl krb5kdc[4268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:28 dist-test-slave-8tdl krb5kdc[4268](info): set up 2 sockets
Aug 08 07:06:28 dist-test-slave-8tdl krb5kdc[4268](info): commencing operation
krb5kdc: starting...
W20260808 07:06:29.081478 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:06:29 dist-test-slave-8tdl krb5kdc[4268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:06:29.243690 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:37935
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:39457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:37935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:29.334486 4281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:29.334811 4281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:29.334879 4281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:29.336681 4281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:29.336763 4281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:29.336789 4281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:29.336820 4281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:29.336851 4281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:29.338760 4281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:39457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:37935
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:37935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:29.339118 4281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:29.339409 4281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:29.344743 4286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:29.344743 4290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:29.344823 4287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:29.345311 4281 server_base.cc:1034] running on GCE node
I20260808 07:06:29.345641 4281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:29.346099 4281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:29.347307 4281 hybrid_clock.cc:648] HybridClock initialized: now 1786172789347233 us; error 87 us; skew 500 ppm
Aug 08 07:06:29 dist-test-slave-8tdl krb5kdc[4268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172789, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:29.350143 4281 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:06:29.350771 4281 webserver.cc:460] Webserver started at http://127.13.207.62:43013/ using document root <none> and password file <none>
I20260808 07:06:29.351035 4281 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:29.351110 4281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:29.351274 4281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:29.352483 4281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "0c5af4865df845568e10358f4bf7d21b"
format_stamp: "Formatted at 2026-08-08 07:06:29 on dist-test-slave-8tdl"
server_key: "16a0c304a10a41c93a9977934208f1b1"
server_key_iv: "52538ee306b690a1086fddbc28931317"
server_key_version: "encryptionkey@0"
I20260808 07:06:29.352885 4281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "0c5af4865df845568e10358f4bf7d21b"
format_stamp: "Formatted at 2026-08-08 07:06:29 on dist-test-slave-8tdl"
server_key: "16a0c304a10a41c93a9977934208f1b1"
server_key_iv: "52538ee306b690a1086fddbc28931317"
server_key_version: "encryptionkey@0"
I20260808 07:06:29.354353 4281 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:06:29.355353 4297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:29.355609 4281 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:06:29.355727 4281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "0c5af4865df845568e10358f4bf7d21b"
format_stamp: "Formatted at 2026-08-08 07:06:29 on dist-test-slave-8tdl"
server_key: "16a0c304a10a41c93a9977934208f1b1"
server_key_iv: "52538ee306b690a1086fddbc28931317"
server_key_version: "encryptionkey@0"
I20260808 07:06:29.355809 4281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:29.370414 4281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:29.373142 4281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:29.373334 4281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:29.378130 4281 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:37935
I20260808 07:06:29.378229 4354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:37935 every 8 connection(s)
I20260808 07:06:29.378660 4281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:29.379783 4355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:29.380996 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 4281
I20260808 07:06:29.381137 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:29.381330 14140 external_mini_cluster.cc:1428] Setting key 3c8ae92e8b206be310b35db96822db9b
I20260808 07:06:29.383553 4355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b: Bootstrap starting.
I20260808 07:06:29.384243 4355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:29.384567 4355 log.cc:824] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:29.385327 4355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b: No bootstrap required, opened a new log
I20260808 07:06:29.387598 4355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c5af4865df845568e10358f4bf7d21b" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 37935 } }
I20260808 07:06:29.387784 4355 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:29.387838 4355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c5af4865df845568e10358f4bf7d21b, State: Initialized, Role: FOLLOWER
I20260808 07:06:29.388010 4355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c5af4865df845568e10358f4bf7d21b" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 37935 } }
I20260808 07:06:29.388115 4355 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:29.388162 4355 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:29.388211 4355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b [term 0 FOLLOWER]: Advancing to term 1
Aug 08 07:06:29 dist-test-slave-8tdl krb5kdc[4268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:06:29.388896 4355 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c5af4865df845568e10358f4bf7d21b" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 37935 } }
I20260808 07:06:29.389022 4355 leader_election.cc:302] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c5af4865df845568e10358f4bf7d21b; no voters:
I20260808 07:06:29.389247 4355 leader_election.cc:288] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:29.389335 4360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:29.389585 4355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:29.389590 4360 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b [term 1 LEADER]: Becoming Leader. State: Replica: 0c5af4865df845568e10358f4bf7d21b, State: Running, Role: LEADER
I20260808 07:06:29.390193 4360 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c5af4865df845568e10358f4bf7d21b" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 37935 } }
I20260808 07:06:29.390727 4362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c5af4865df845568e10358f4bf7d21b. Latest consensus state: current_term: 1 leader_uuid: "0c5af4865df845568e10358f4bf7d21b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c5af4865df845568e10358f4bf7d21b" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 37935 } } }
I20260808 07:06:29.390832 4362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:29.390949 4361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c5af4865df845568e10358f4bf7d21b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c5af4865df845568e10358f4bf7d21b" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 37935 } } }
I20260808 07:06:29.391044 4361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b [sys.catalog]: This master's current role is: LEADER
W20260808 07:06:29.391479 4375 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:06:29.391538 4375 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:06:29.391824 4374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:29.392027 4374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:29.393515 4358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:29.381934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55771 (local address 127.13.207.62:37935)
0808 07:06:29.382157 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:29.382172 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:29.382199 (+ 27us) server_negotiation.cc:407] Connection header received
0808 07:06:29.382649 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:29.382656 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:29.382768 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:29.382935 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:29.383610 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:29.384717 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:29.385686 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:29.386125 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:29.386259 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:29.389143 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:29.389147 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:29.389148 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:29.389162 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:29.391617 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:29.392172 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:29.392175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:29.392176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:29.392278 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:29.392550 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:29.392551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:29.392552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:29.392866 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:29.393065 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:29.393197 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 07:06:29.393303 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":105,"threads_started":1}
I20260808 07:06:29.394491 4374 catalog_manager.cc:1347] Generated new cluster ID: 4cb1dd2d33e14383845a71877ca78b1d
I20260808 07:06:29.394541 4374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:29.394604 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:29.402233 4374 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:29.403064 4374 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:29.407374 4374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c5af4865df845568e10358f4bf7d21b: Generated new TSK 0
I20260808 07:06:29.407727 4374 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:29.411190 4358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:29.396400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59365 (local address 127.13.207.62:37935)
0808 07:06:29.396664 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:29.396666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:29.402058 (+ 5392us) server_negotiation.cc:407] Connection header received
0808 07:06:29.402137 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:29.402138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:29.402150 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:29.402206 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:29.402970 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:29.403547 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:29.404271 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:29.404536 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:29.404612 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:29.406258 (+ 1646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:29.406259 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:29.406260 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:29.406270 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:29.408502 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:29.409395 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:29.409397 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:29.409397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:29.409479 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:29.410461 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:29.410462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:29.410463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:29.410720 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:29.410874 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:29.410998 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 07:06:29.411081 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":239}
I20260808 07:06:29.415254 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 4281
W20260808 07:06:29.422173 4266 connection.cc:535] client connection to 127.13.207.62:37935 recv error: Network error: failed to read from TLS socket (remote: 127.13.207.62:37935): Connection reset by peer (error 104)
2026-08-08T07:06:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1459 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:29 dist-test-slave-8tdl krb5kdc[4386](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:29 dist-test-slave-8tdl krb5kdc[4386](info): set up 2 sockets
Aug 08 07:06:29 dist-test-slave-8tdl krb5kdc[4386](info): commencing operation
krb5kdc: starting...
W20260808 07:06:30.523406 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:06:30 dist-test-slave-8tdl krb5kdc[4386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:06:30.666453 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:40243
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:36455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:40243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:30.755242 4400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:30.755553 4400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:30.755604 4400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:30.756958 4400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:30.757052 4400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:30.757078 4400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:30.757102 4400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:30.757133 4400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:30.758796 4400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:36455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:40243
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:40243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:30.759106 4400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:30.759368 4400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:30.764312 4408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:30.764529 4406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:30.764525 4405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:30.765573 4400 server_base.cc:1034] running on GCE node
I20260808 07:06:30.765955 4400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:30.766252 4400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:30.767452 4400 hybrid_clock.cc:648] HybridClock initialized: now 1786172790767443 us; error 43 us; skew 500 ppm
Aug 08 07:06:30 dist-test-slave-8tdl krb5kdc[4386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172790, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:30.776763 4400 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:06:30.777357 4400 webserver.cc:460] Webserver started at http://127.13.207.62:43689/ using document root <none> and password file <none>
I20260808 07:06:30.777650 4400 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:30.777729 4400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:30.777892 4400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:30.779166 4400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "50a413da285d4df6a25ea051076d359a"
format_stamp: "Formatted at 2026-08-08 07:06:30 on dist-test-slave-8tdl"
server_key: "30aaf02a9ae72a9f891f6a7b36b6ce47"
server_key_iv: "907f23f4f073f8c67e83a3190b88e1eb"
server_key_version: "encryptionkey@0"
I20260808 07:06:30.779554 4400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "50a413da285d4df6a25ea051076d359a"
format_stamp: "Formatted at 2026-08-08 07:06:30 on dist-test-slave-8tdl"
server_key: "30aaf02a9ae72a9f891f6a7b36b6ce47"
server_key_iv: "907f23f4f073f8c67e83a3190b88e1eb"
server_key_version: "encryptionkey@0"
I20260808 07:06:30.781243 4400 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:06:30.782299 4415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:30.782657 4400 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:30.782877 4400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "50a413da285d4df6a25ea051076d359a"
format_stamp: "Formatted at 2026-08-08 07:06:30 on dist-test-slave-8tdl"
server_key: "30aaf02a9ae72a9f891f6a7b36b6ce47"
server_key_iv: "907f23f4f073f8c67e83a3190b88e1eb"
server_key_version: "encryptionkey@0"
I20260808 07:06:30.783053 4400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:30.821909 4400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:30.824440 4400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:30.824563 4400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:30.828732 4400 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:40243
I20260808 07:06:30.828824 4470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:40243 every 8 connection(s)
I20260808 07:06:30.829192 4400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:30.830216 4471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:30.832651 4471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a: Bootstrap starting.
I20260808 07:06:30.833279 4471 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:30.833591 4471 log.cc:824] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:30.833932 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 4400
I20260808 07:06:30.834097 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:30.834319 14140 external_mini_cluster.cc:1428] Setting key 1a80da00b0cd00b5a33540511c9ce46d
I20260808 07:06:30.834482 4471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a: No bootstrap required, opened a new log
I20260808 07:06:30.836668 4471 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a413da285d4df6a25ea051076d359a" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40243 } }
I20260808 07:06:30.836824 4471 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:30.836848 4471 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50a413da285d4df6a25ea051076d359a, State: Initialized, Role: FOLLOWER
I20260808 07:06:30.836992 4471 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a413da285d4df6a25ea051076d359a" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40243 } }
I20260808 07:06:30.837085 4471 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:30.837126 4471 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:30.837180 4471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:30.838092 4471 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a413da285d4df6a25ea051076d359a" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40243 } }
I20260808 07:06:30.838217 4471 leader_election.cc:302] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50a413da285d4df6a25ea051076d359a; no voters:
I20260808 07:06:30.838389 4471 leader_election.cc:288] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:30.838461 4476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:30.838644 4471 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:30.838708 4476 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a [term 1 LEADER]: Becoming Leader. State: Replica: 50a413da285d4df6a25ea051076d359a, State: Running, Role: LEADER
I20260808 07:06:30.838963 4476 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a413da285d4df6a25ea051076d359a" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40243 } }
I20260808 07:06:30.839444 4477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50a413da285d4df6a25ea051076d359a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a413da285d4df6a25ea051076d359a" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40243 } } }
I20260808 07:06:30.839535 4477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:30.839941 4478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50a413da285d4df6a25ea051076d359a. Latest consensus state: current_term: 1 leader_uuid: "50a413da285d4df6a25ea051076d359a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a413da285d4df6a25ea051076d359a" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 40243 } } }
I20260808 07:06:30.840025 4478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a [sys.catalog]: This master's current role is: LEADER
W20260808 07:06:30.840787 4493 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:06:30.840862 4493 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:06:30.840965 4486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:30.841192 4486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:30.842941 4486 catalog_manager.cc:1347] Generated new cluster ID: 8b66a4b2f0f04f10b0bda9de403c4b3b
I20260808 07:06:30.843007 4486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:30.843259 4475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:30.835009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59153 (local address 127.13.207.62:40243)
0808 07:06:30.835221 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:30.835231 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:30.835267 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:06:30.836673 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:30.836688 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:30.836762 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:30.837016 (+ 254us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:30.837289 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:30.839099 (+ 1810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:30.840486 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:30.841023 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:30.841134 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:30.842215 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:30.842220 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:30.842220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:30.842456 (+ 236us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:30.842502 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:30.842520 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:30.842617 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:30.842985 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 07:06:30.843017 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":109,"threads_started":1}
I20260808 07:06:30.844396 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:30.850204 4475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:30.846220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46121 (local address 127.13.207.62:40243)
0808 07:06:30.846284 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:30.846286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:30.846295 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:06:30.846404 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:30.846406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:30.846415 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:30.846465 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:30.846626 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:30.847390 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:30.848689 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:30.849022 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:30.849081 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:30.849903 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:30.849905 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:30.849905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:30.849910 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:30.849925 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:30.849930 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:30.849979 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:30.850101 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 07:06:30.850115 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:06:30.872431 4486 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:30.873351 4486 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:30.882831 4486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50a413da285d4df6a25ea051076d359a: Generated new TSK 0
I20260808 07:06:30.883108 4486 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:30.894857 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 4400
W20260808 07:06:30.900585 4384 connection.cc:535] client connection to 127.13.207.62:40243 recv error: Network error: failed to read from TLS socket (remote: 127.13.207.62:40243): Connection reset by peer (error 104)
2026-08-08T07:06:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1478 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:30 dist-test-slave-8tdl krb5kdc[4504](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:30 dist-test-slave-8tdl krb5kdc[4504](info): set up 2 sockets
Aug 08 07:06:30 dist-test-slave-8tdl krb5kdc[4504](info): commencing operation
krb5kdc: starting...
W20260808 07:06:32.049309 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:06:32 dist-test-slave-8tdl krb5kdc[4504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172792, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:06:32.219643 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:32957
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:34939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:32957
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:32.315820 4517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:32.316067 4517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:32.316095 4517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:32.317901 4517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:32.317970 4517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:32.317986 4517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:32.317998 4517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:32.318011 4517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:32.319783 4517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:34939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:32957
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:32957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:32.320086 4517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:32.320376 4517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:32.325945 4522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:32.326036 4517 server_base.cc:1034] running on GCE node
W20260808 07:06:32.326030 4523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:32.326009 4526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:32.326576 4517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:32.326970 4517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:32.328194 4517 hybrid_clock.cc:648] HybridClock initialized: now 1786172792328166 us; error 56 us; skew 500 ppm
Aug 08 07:06:32 dist-test-slave-8tdl krb5kdc[4504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172792, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:32.331235 4517 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:06:32.331867 4517 webserver.cc:460] Webserver started at http://127.13.207.62:39861/ using document root <none> and password file <none>
I20260808 07:06:32.332158 4517 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:32.332239 4517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:32.332479 4517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:32.333781 4517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "cfbfce8f4f5d43e7bb11dec2bf4ceebc"
format_stamp: "Formatted at 2026-08-08 07:06:32 on dist-test-slave-8tdl"
server_key: "8d730614249e926457f032dc5258feb3"
server_key_iv: "6d8687d78b771823dce90bb905a8f89d"
server_key_version: "encryptionkey@0"
I20260808 07:06:32.334223 4517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "cfbfce8f4f5d43e7bb11dec2bf4ceebc"
format_stamp: "Formatted at 2026-08-08 07:06:32 on dist-test-slave-8tdl"
server_key: "8d730614249e926457f032dc5258feb3"
server_key_iv: "6d8687d78b771823dce90bb905a8f89d"
server_key_version: "encryptionkey@0"
I20260808 07:06:32.336105 4517 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 07:06:32.337162 4533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:32.337447 4517 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:32.337594 4517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "cfbfce8f4f5d43e7bb11dec2bf4ceebc"
format_stamp: "Formatted at 2026-08-08 07:06:32 on dist-test-slave-8tdl"
server_key: "8d730614249e926457f032dc5258feb3"
server_key_iv: "6d8687d78b771823dce90bb905a8f89d"
server_key_version: "encryptionkey@0"
I20260808 07:06:32.337693 4517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:32.341872 4517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:32.344606 4517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:32.344887 4517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:32.349938 4517 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:32957
I20260808 07:06:32.349978 4585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:32957 every 8 connection(s)
I20260808 07:06:32.350392 4517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:32.351486 4586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:32.353844 4586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfbfce8f4f5d43e7bb11dec2bf4ceebc: Bootstrap starting.
I20260808 07:06:32.354496 4586 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfbfce8f4f5d43e7bb11dec2bf4ceebc: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:32.354861 4586 log.cc:824] T 00000000000000000000000000000000 P cfbfce8f4f5d43e7bb11dec2bf4ceebc: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:32.355834 4586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfbfce8f4f5d43e7bb11dec2bf4ceebc: No bootstrap required, opened a new log
I20260808 07:06:32.356818 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 4517
I20260808 07:06:32.356951 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:32.357154 14140 external_mini_cluster.cc:1428] Setting key a7592c3e0eb4b84e7dda18f67872d499
I20260808 07:06:32.358234 4586 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfbfce8f4f5d43e7bb11dec2bf4ceebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbfce8f4f5d43e7bb11dec2bf4ceebc" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 32957 } }
I20260808 07:06:32.358407 4586 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfbfce8f4f5d43e7bb11dec2bf4ceebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:32.358475 4586 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfbfce8f4f5d43e7bb11dec2bf4ceebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfbfce8f4f5d43e7bb11dec2bf4ceebc, State: Initialized, Role: FOLLOWER
I20260808 07:06:32.358668 4586 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfbfce8f4f5d43e7bb11dec2bf4ceebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbfce8f4f5d43e7bb11dec2bf4ceebc" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 32957 } }
I20260808 07:06:32.358809 4586 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfbfce8f4f5d43e7bb11dec2bf4ceebc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:32.358842 4586 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfbfce8f4f5d43e7bb11dec2bf4ceebc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:32.358889 4586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfbfce8f4f5d43e7bb11dec2bf4ceebc [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:32.359711 4590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:32.357767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49401 (local address 127.13.207.62:32957)
0808 07:06:32.357991 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:32.357999 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:32.358027 (+ 28us) server_negotiation.cc:407] Connection header received
0808 07:06:32.358215 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:32.358221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:32.358248 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:32.358428 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:32.359209 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:32.359213 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:32.359213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:32.359218 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:32.359270 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:32.359282 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:32.359362 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:32.359443 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 07:06:32.359489 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":103,"threads_started":1}
I20260808 07:06:32.359920 4586 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfbfce8f4f5d43e7bb11dec2bf4ceebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbfce8f4f5d43e7bb11dec2bf4ceebc" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 32957 } }
I20260808 07:06:32.360064 4586 leader_election.cc:302] T 00000000000000000000000000000000 P cfbfce8f4f5d43e7bb11dec2bf4ceebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfbfce8f4f5d43e7bb11dec2bf4ceebc; no voters:
I20260808 07:06:32.360239 4586 leader_election.cc:288] T 00000000000000000000000000000000 P cfbfce8f4f5d43e7bb11dec2bf4ceebc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:32.360301 4591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfbfce8f4f5d43e7bb11dec2bf4ceebc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:32.360491 4591 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfbfce8f4f5d43e7bb11dec2bf4ceebc [term 1 LEADER]: Becoming Leader. State: Replica: cfbfce8f4f5d43e7bb11dec2bf4ceebc, State: Running, Role: LEADER
I20260808 07:06:32.360529 4586 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfbfce8f4f5d43e7bb11dec2bf4ceebc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:32.360581 4591 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfbfce8f4f5d43e7bb11dec2bf4ceebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbfce8f4f5d43e7bb11dec2bf4ceebc" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 32957 } }
I20260808 07:06:32.360791 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:32.360930 4591 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfbfce8f4f5d43e7bb11dec2bf4ceebc [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfbfce8f4f5d43e7bb11dec2bf4ceebc. Latest consensus state: current_term: 1 leader_uuid: "cfbfce8f4f5d43e7bb11dec2bf4ceebc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbfce8f4f5d43e7bb11dec2bf4ceebc" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 32957 } } }
I20260808 07:06:32.360939 4593 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfbfce8f4f5d43e7bb11dec2bf4ceebc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfbfce8f4f5d43e7bb11dec2bf4ceebc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbfce8f4f5d43e7bb11dec2bf4ceebc" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 32957 } } }
I20260808 07:06:32.361017 4591 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfbfce8f4f5d43e7bb11dec2bf4ceebc [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:32.361018 4593 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfbfce8f4f5d43e7bb11dec2bf4ceebc [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:32.361775 4598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:32.362445 4598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:32.363978 4590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:32.362567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60763 (local address 127.13.207.62:32957)
0808 07:06:32.362603 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:32.362605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:32.362956 (+ 351us) server_negotiation.cc:407] Connection header received
0808 07:06:32.363004 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:32.363004 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:32.363037 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:32.363103 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:32.363724 (+ 621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:32.363726 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:32.363726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:32.363730 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:32.363746 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:32.363751 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:32.363816 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:32.363880 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 07:06:32.363895 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:06:32.364498 4598 catalog_manager.cc:1347] Generated new cluster ID: ea695a1904954ef0a2a8c0f311c1f9c7
I20260808 07:06:32.364573 4598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:32.371538 4598 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:32.371773 4598 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:32.378139 4598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfbfce8f4f5d43e7bb11dec2bf4ceebc: Generated new TSK 0
I20260808 07:06:32.378371 4598 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:32.382594 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 4517
2026-08-08T07:06:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1488 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:32 dist-test-slave-8tdl krb5kdc[4622](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:32 dist-test-slave-8tdl krb5kdc[4622](info): set up 2 sockets
Aug 08 07:06:32 dist-test-slave-8tdl krb5kdc[4622](info): commencing operation
krb5kdc: starting...
W20260808 07:06:33.511197 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:06:33 dist-test-slave-8tdl krb5kdc[4622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:06:33.659000 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:42415
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:41129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:42415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:33.756209 4636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:33.756516 4636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:33.756565 4636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:33.758069 4636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:33.758167 4636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:33.758193 4636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:33.758215 4636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:33.758246 4636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:33.760070 4636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:41129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:42415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:42415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:33.760394 4636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:33.760661 4636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:33.766366 4644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:33.766373 4642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:33.766592 4636 server_base.cc:1034] running on GCE node
W20260808 07:06:33.766486 4641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:33.767031 4636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:33.767428 4636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:33.768635 4636 hybrid_clock.cc:648] HybridClock initialized: now 1786172793768615 us; error 42 us; skew 500 ppm
Aug 08 07:06:33 dist-test-slave-8tdl krb5kdc[4622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172793, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:33.771848 4636 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:06:33.772415 4636 webserver.cc:460] Webserver started at http://127.13.207.62:33387/ using document root <none> and password file <none>
I20260808 07:06:33.772662 4636 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:33.772712 4636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:33.772854 4636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:33.774040 4636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "cf3b5ad7d4a74017bb057465af230f81"
format_stamp: "Formatted at 2026-08-08 07:06:33 on dist-test-slave-8tdl"
server_key: "f4438781e165f6cc0dbb5ec6f980db2d"
server_key_iv: "bb11a529ece37b1911dfebf6b932a801"
server_key_version: "encryptionkey@0"
I20260808 07:06:33.774435 4636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "cf3b5ad7d4a74017bb057465af230f81"
format_stamp: "Formatted at 2026-08-08 07:06:33 on dist-test-slave-8tdl"
server_key: "f4438781e165f6cc0dbb5ec6f980db2d"
server_key_iv: "bb11a529ece37b1911dfebf6b932a801"
server_key_version: "encryptionkey@0"
I20260808 07:06:33.776052 4636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:06:33.776904 4652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:33.777144 4636 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:33.777263 4636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "cf3b5ad7d4a74017bb057465af230f81"
format_stamp: "Formatted at 2026-08-08 07:06:33 on dist-test-slave-8tdl"
server_key: "f4438781e165f6cc0dbb5ec6f980db2d"
server_key_iv: "bb11a529ece37b1911dfebf6b932a801"
server_key_version: "encryptionkey@0"
I20260808 07:06:33.777347 4636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:33.794611 4636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:33.797564 4636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:33.797778 4636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:33.802798 4636 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:42415
I20260808 07:06:33.802845 4705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:42415 every 8 connection(s)
I20260808 07:06:33.803251 4636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:33.804316 4706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:33.806162 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 4636
I20260808 07:06:33.806298 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:33.806493 14140 external_mini_cluster.cc:1428] Setting key de69adabcb4fdce6279174ecd3aaf107
I20260808 07:06:33.807344 4706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf3b5ad7d4a74017bb057465af230f81: Bootstrap starting.
I20260808 07:06:33.807999 4706 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf3b5ad7d4a74017bb057465af230f81: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:33.808323 4706 log.cc:824] T 00000000000000000000000000000000 P cf3b5ad7d4a74017bb057465af230f81: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:33.809266 4706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf3b5ad7d4a74017bb057465af230f81: No bootstrap required, opened a new log
I20260808 07:06:33.811342 4706 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf3b5ad7d4a74017bb057465af230f81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3b5ad7d4a74017bb057465af230f81" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 42415 } }
I20260808 07:06:33.811533 4706 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf3b5ad7d4a74017bb057465af230f81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:33.811604 4706 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf3b5ad7d4a74017bb057465af230f81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf3b5ad7d4a74017bb057465af230f81, State: Initialized, Role: FOLLOWER
I20260808 07:06:33.811808 4706 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf3b5ad7d4a74017bb057465af230f81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3b5ad7d4a74017bb057465af230f81" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 42415 } }
I20260808 07:06:33.811945 4706 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf3b5ad7d4a74017bb057465af230f81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:33.811983 4706 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf3b5ad7d4a74017bb057465af230f81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:33.812027 4706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf3b5ad7d4a74017bb057465af230f81 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:33.812765 4706 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf3b5ad7d4a74017bb057465af230f81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3b5ad7d4a74017bb057465af230f81" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 42415 } }
I20260808 07:06:33.812916 4706 leader_election.cc:302] T 00000000000000000000000000000000 P cf3b5ad7d4a74017bb057465af230f81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf3b5ad7d4a74017bb057465af230f81; no voters:
I20260808 07:06:33.813158 4706 leader_election.cc:288] T 00000000000000000000000000000000 P cf3b5ad7d4a74017bb057465af230f81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:33.813189 4711 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf3b5ad7d4a74017bb057465af230f81 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:33.813313 4711 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf3b5ad7d4a74017bb057465af230f81 [term 1 LEADER]: Becoming Leader. State: Replica: cf3b5ad7d4a74017bb057465af230f81, State: Running, Role: LEADER
I20260808 07:06:33.813472 4706 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf3b5ad7d4a74017bb057465af230f81 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:33.813460 4711 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf3b5ad7d4a74017bb057465af230f81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3b5ad7d4a74017bb057465af230f81" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 42415 } }
I20260808 07:06:33.813995 4713 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf3b5ad7d4a74017bb057465af230f81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf3b5ad7d4a74017bb057465af230f81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3b5ad7d4a74017bb057465af230f81" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 42415 } } }
I20260808 07:06:33.814265 4713 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf3b5ad7d4a74017bb057465af230f81 [sys.catalog]: This master's current role is: LEADER
Aug 08 07:06:33 dist-test-slave-8tdl krb5kdc[4622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:06:33.814179 4712 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf3b5ad7d4a74017bb057465af230f81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf3b5ad7d4a74017bb057465af230f81. Latest consensus state: current_term: 1 leader_uuid: "cf3b5ad7d4a74017bb057465af230f81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf3b5ad7d4a74017bb057465af230f81" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 42415 } } }
I20260808 07:06:33.814836 4712 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf3b5ad7d4a74017bb057465af230f81 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:33.815013 4721 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:33.815591 4721 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:33.817389 4721 catalog_manager.cc:1347] Generated new cluster ID: 76b97ff9ff7545b8a4cbe53a1431822f
I20260808 07:06:33.817451 4721 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:33.819779 4709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:33.807143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54507 (local address 127.13.207.62:42415)
0808 07:06:33.807352 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:33.807364 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:33.807414 (+ 50us) server_negotiation.cc:407] Connection header received
0808 07:06:33.807910 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:33.807918 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:33.808028 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:33.808208 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:33.809105 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:33.810130 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:33.811032 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:33.811474 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:33.811601 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:33.815273 (+ 3672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:33.815278 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:33.815278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:33.815294 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:33.817634 (+ 2340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:33.818157 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:33.818161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:33.818161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:33.818296 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:33.818523 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:33.818524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:33.818524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:33.818944 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:33.819140 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:33.819233 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 07:06:33.819414 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":93,"threads_started":1}
I20260808 07:06:33.821718 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:33.831687 4709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:33.823656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33801 (local address 127.13.207.62:42415)
0808 07:06:33.823697 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:33.823699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:33.823828 (+ 129us) server_negotiation.cc:407] Connection header received
0808 07:06:33.823905 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:33.823906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:33.823916 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:33.823983 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:33.824757 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:33.825611 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:33.826764 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:33.827091 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:33.827157 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:33.828717 (+ 1560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:33.828718 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:33.828719 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:33.828727 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:33.830372 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:33.830846 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:33.830847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:33.830847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:33.830920 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:33.831136 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:33.831137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:33.831137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:33.831269 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:33.831359 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:33.831498 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 07:06:33.831587 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:06:33.844131 4721 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:33.844805 4721 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:33.860163 4721 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf3b5ad7d4a74017bb057465af230f81: Generated new TSK 0
I20260808 07:06:33.860425 4721 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:33.866328 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 4636
2026-08-08T07:06:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1482 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:33 dist-test-slave-8tdl krb5kdc[4738](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:33 dist-test-slave-8tdl krb5kdc[4738](info): set up 2 sockets
Aug 08 07:06:33 dist-test-slave-8tdl krb5kdc[4738](info): commencing operation
krb5kdc: starting...
W20260808 07:06:34.995779 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:06:35 dist-test-slave-8tdl krb5kdc[4738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:06:35.152554 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:43915
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:45839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:43915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:35.248793 4751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:35.249135 4751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:35.249171 4751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:35.250974 4751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:35.251060 4751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:35.251078 4751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:35.251094 4751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:35.251108 4751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:35.253424 4751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:45839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:43915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:43915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:35.253765 4751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:35.254081 4751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:35.260272 4757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:35.260272 4759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:35.260627 4756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:35.260800 4751 server_base.cc:1034] running on GCE node
I20260808 07:06:35.261090 4751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:35.261426 4751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:35.262656 4751 hybrid_clock.cc:648] HybridClock initialized: now 1786172795262608 us; error 69 us; skew 500 ppm
Aug 08 07:06:35 dist-test-slave-8tdl krb5kdc[4738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172795, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:35.265758 4751 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:06:35.266286 4751 webserver.cc:460] Webserver started at http://127.13.207.62:35613/ using document root <none> and password file <none>
I20260808 07:06:35.266571 4751 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:35.266645 4751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:35.266858 4751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:35.268038 4751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "8208b6eeffdd49999f1915c40a215cb5"
format_stamp: "Formatted at 2026-08-08 07:06:35 on dist-test-slave-8tdl"
server_key: "1771e169ab3cd6f67e5693f22410de25"
server_key_iv: "db6075049a707cb9e1e4740def629e9b"
server_key_version: "encryptionkey@0"
I20260808 07:06:35.268429 4751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "8208b6eeffdd49999f1915c40a215cb5"
format_stamp: "Formatted at 2026-08-08 07:06:35 on dist-test-slave-8tdl"
server_key: "1771e169ab3cd6f67e5693f22410de25"
server_key_iv: "db6075049a707cb9e1e4740def629e9b"
server_key_version: "encryptionkey@0"
I20260808 07:06:35.269980 4751 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:06:35.271167 4767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:35.271510 4751 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:35.271642 4751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "8208b6eeffdd49999f1915c40a215cb5"
format_stamp: "Formatted at 2026-08-08 07:06:35 on dist-test-slave-8tdl"
server_key: "1771e169ab3cd6f67e5693f22410de25"
server_key_iv: "db6075049a707cb9e1e4740def629e9b"
server_key_version: "encryptionkey@0"
I20260808 07:06:35.271729 4751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:35.292457 4751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:35.295006 4751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:35.295207 4751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:35.299784 4751 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:43915
I20260808 07:06:35.299827 4824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:43915 every 8 connection(s)
I20260808 07:06:35.300207 4751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:35.301280 4825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:35.304198 4825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8208b6eeffdd49999f1915c40a215cb5: Bootstrap starting.
I20260808 07:06:35.304932 4825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8208b6eeffdd49999f1915c40a215cb5: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:35.305263 4825 log.cc:824] T 00000000000000000000000000000000 P 8208b6eeffdd49999f1915c40a215cb5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:35.306063 4825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8208b6eeffdd49999f1915c40a215cb5: No bootstrap required, opened a new log
I20260808 07:06:35.308064 4825 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8208b6eeffdd49999f1915c40a215cb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8208b6eeffdd49999f1915c40a215cb5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 43915 } }
I20260808 07:06:35.308229 4825 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8208b6eeffdd49999f1915c40a215cb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:35.308256 4825 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8208b6eeffdd49999f1915c40a215cb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8208b6eeffdd49999f1915c40a215cb5, State: Initialized, Role: FOLLOWER
I20260808 07:06:35.308425 4825 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8208b6eeffdd49999f1915c40a215cb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8208b6eeffdd49999f1915c40a215cb5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 43915 } }
I20260808 07:06:35.308512 4825 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8208b6eeffdd49999f1915c40a215cb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:35.308537 4825 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8208b6eeffdd49999f1915c40a215cb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:35.308570 4825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8208b6eeffdd49999f1915c40a215cb5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:35.309511 4825 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8208b6eeffdd49999f1915c40a215cb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8208b6eeffdd49999f1915c40a215cb5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 43915 } }
I20260808 07:06:35.309664 4825 leader_election.cc:302] T 00000000000000000000000000000000 P 8208b6eeffdd49999f1915c40a215cb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8208b6eeffdd49999f1915c40a215cb5; no voters:
I20260808 07:06:35.309811 4825 leader_election.cc:288] T 00000000000000000000000000000000 P 8208b6eeffdd49999f1915c40a215cb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:35.309918 4828 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8208b6eeffdd49999f1915c40a215cb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:35.310113 4825 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8208b6eeffdd49999f1915c40a215cb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:35.310143 4828 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8208b6eeffdd49999f1915c40a215cb5 [term 1 LEADER]: Becoming Leader. State: Replica: 8208b6eeffdd49999f1915c40a215cb5, State: Running, Role: LEADER
I20260808 07:06:35.310245 4828 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8208b6eeffdd49999f1915c40a215cb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8208b6eeffdd49999f1915c40a215cb5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 43915 } }
I20260808 07:06:35.310456 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 4751
I20260808 07:06:35.310595 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:35.310740 4829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8208b6eeffdd49999f1915c40a215cb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8208b6eeffdd49999f1915c40a215cb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8208b6eeffdd49999f1915c40a215cb5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 43915 } } }
I20260808 07:06:35.310788 4830 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8208b6eeffdd49999f1915c40a215cb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8208b6eeffdd49999f1915c40a215cb5. Latest consensus state: current_term: 1 leader_uuid: "8208b6eeffdd49999f1915c40a215cb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8208b6eeffdd49999f1915c40a215cb5" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 43915 } } }
I20260808 07:06:35.310834 4829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8208b6eeffdd49999f1915c40a215cb5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:35.310855 14140 external_mini_cluster.cc:1428] Setting key 3d5bcb438116fcdc547cb9d80e3af40f
I20260808 07:06:35.310835 4830 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8208b6eeffdd49999f1915c40a215cb5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:35.311116 4837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:35.312011 4837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:35.313828 4837 catalog_manager.cc:1347] Generated new cluster ID: 54cddc4e658d47a8b105df641d820a70
I20260808 07:06:35.313889 4837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:35.317576 4847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:35.311634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51405 (local address 127.13.207.62:43915)
0808 07:06:35.311868 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:35.311879 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:35.311945 (+ 66us) server_negotiation.cc:407] Connection header received
0808 07:06:35.312157 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:35.312167 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:35.312244 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:35.312692 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:35.312808 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:35.314215 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:35.315271 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:35.315779 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:35.315916 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:35.316749 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:35.316754 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:35.316754 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:35.316997 (+ 243us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:35.317045 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:35.317065 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:35.317144 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:35.317247 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 07:06:35.317325 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":72,"threads_started":1}
I20260808 07:06:35.318602 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:35.325591 4837 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:35.326324 4837 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:35.328496 4847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:35.320178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50949 (local address 127.13.207.62:43915)
0808 07:06:35.320221 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:35.320224 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:35.324929 (+ 4705us) server_negotiation.cc:407] Connection header received
0808 07:06:35.324978 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:35.324978 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:35.324993 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:35.325081 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:35.325484 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:35.326194 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:35.327126 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:35.327359 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:35.327437 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:35.328181 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:35.328183 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:35.328183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:35.328189 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:35.328205 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:35.328215 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:35.328276 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:35.328347 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 07:06:35.328391 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:06:35.329385 4837 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8208b6eeffdd49999f1915c40a215cb5: Generated new TSK 0
I20260808 07:06:35.329589 4837 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:35.332353 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 4751
2026-08-08T07:06:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1466 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:35 dist-test-slave-8tdl krb5kdc[4857](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:35 dist-test-slave-8tdl krb5kdc[4857](info): set up 2 sockets
Aug 08 07:06:35 dist-test-slave-8tdl krb5kdc[4857](info): commencing operation
krb5kdc: starting...
W20260808 07:06:36.459924 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:06:36 dist-test-slave-8tdl krb5kdc[4857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172796, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:06:36.608031 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:33501
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:41827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:33501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:36.698531 4871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:36.698850 4871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:36.698882 4871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:36.700335 4871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:36.700415 4871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:36.700431 4871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:36.700445 4871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:36.700457 4871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:36.702344 4871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:41827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:33501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:33501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:36.702667 4871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:36.702986 4871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:36.708622 4876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:36.708622 4877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:36.708942 4879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:36.709942 4871 server_base.cc:1034] running on GCE node
I20260808 07:06:36.710242 4871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:36.710768 4871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:36.712000 4871 hybrid_clock.cc:648] HybridClock initialized: now 1786172796711966 us; error 47 us; skew 500 ppm
Aug 08 07:06:36 dist-test-slave-8tdl krb5kdc[4857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172796, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:36.721616 4871 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:06:36.722407 4871 webserver.cc:460] Webserver started at http://127.13.207.62:35937/ using document root <none> and password file <none>
I20260808 07:06:36.722774 4871 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:36.723248 4871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:36.723433 4871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:36.724692 4871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "7c96acf1a2c24951a47b7bf1b5b6f445"
format_stamp: "Formatted at 2026-08-08 07:06:36 on dist-test-slave-8tdl"
server_key: "a0b4ca9a06bad203b1a531991f1bee2b"
server_key_iv: "b01897a090a13c05cfbc45727bfdf2ec"
server_key_version: "encryptionkey@0"
I20260808 07:06:36.725126 4871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "7c96acf1a2c24951a47b7bf1b5b6f445"
format_stamp: "Formatted at 2026-08-08 07:06:36 on dist-test-slave-8tdl"
server_key: "a0b4ca9a06bad203b1a531991f1bee2b"
server_key_iv: "b01897a090a13c05cfbc45727bfdf2ec"
server_key_version: "encryptionkey@0"
I20260808 07:06:36.727200 4871 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260808 07:06:36.728459 4887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:36.728766 4871 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:36.728991 4871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "7c96acf1a2c24951a47b7bf1b5b6f445"
format_stamp: "Formatted at 2026-08-08 07:06:36 on dist-test-slave-8tdl"
server_key: "a0b4ca9a06bad203b1a531991f1bee2b"
server_key_iv: "b01897a090a13c05cfbc45727bfdf2ec"
server_key_version: "encryptionkey@0"
I20260808 07:06:36.729100 4871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:36.732877 4871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:36.736667 4871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:36.736923 4871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:36.743010 4871 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:33501
I20260808 07:06:36.743032 4942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:33501 every 8 connection(s)
I20260808 07:06:36.743566 4871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:36.744863 4943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:36.746253 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 4871
I20260808 07:06:36.746513 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:36.746802 14140 external_mini_cluster.cc:1428] Setting key 8a9ee0b02c90f8299b8f1bb33531c401
I20260808 07:06:36.748266 4943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c96acf1a2c24951a47b7bf1b5b6f445: Bootstrap starting.
I20260808 07:06:36.749337 4943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c96acf1a2c24951a47b7bf1b5b6f445: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:36.749864 4943 log.cc:824] T 00000000000000000000000000000000 P 7c96acf1a2c24951a47b7bf1b5b6f445: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:36.750510 4946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:36.747502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48633 (local address 127.13.207.62:33501)
0808 07:06:36.747714 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:36.747723 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:36.747753 (+ 30us) server_negotiation.cc:407] Connection header received
0808 07:06:36.748296 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:36.748304 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:36.748379 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:36.748544 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:36.749699 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:36.749704 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:36.749704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:36.749711 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:36.749771 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:36.749920 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:36.750085 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:36.750220 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 07:06:36.750271 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":105,"threads_started":1}
I20260808 07:06:36.751147 4943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c96acf1a2c24951a47b7bf1b5b6f445: No bootstrap required, opened a new log
I20260808 07:06:36.751668 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:36.754171 4943 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c96acf1a2c24951a47b7bf1b5b6f445 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c96acf1a2c24951a47b7bf1b5b6f445" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33501 } }
I20260808 07:06:36.754400 4943 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c96acf1a2c24951a47b7bf1b5b6f445 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:36.754477 4943 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c96acf1a2c24951a47b7bf1b5b6f445 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c96acf1a2c24951a47b7bf1b5b6f445, State: Initialized, Role: FOLLOWER
I20260808 07:06:36.754629 4943 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c96acf1a2c24951a47b7bf1b5b6f445 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c96acf1a2c24951a47b7bf1b5b6f445" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33501 } }
I20260808 07:06:36.754765 4946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:36.753277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40793 (local address 127.13.207.62:33501)
0808 07:06:36.753329 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:36.753331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:36.753464 (+ 133us) server_negotiation.cc:407] Connection header received
0808 07:06:36.753592 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:36.753593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:36.753599 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:36.753665 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:36.754370 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:36.754371 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:36.754371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:36.754375 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:36.754390 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:36.754393 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:36.754574 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:36.754642 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 07:06:36.754659 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 07:06:36.754798 4943 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c96acf1a2c24951a47b7bf1b5b6f445 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:36.754859 4943 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c96acf1a2c24951a47b7bf1b5b6f445 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:36.754911 4943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c96acf1a2c24951a47b7bf1b5b6f445 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:36.755703 4943 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c96acf1a2c24951a47b7bf1b5b6f445 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c96acf1a2c24951a47b7bf1b5b6f445" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33501 } }
I20260808 07:06:36.755854 4943 leader_election.cc:302] T 00000000000000000000000000000000 P 7c96acf1a2c24951a47b7bf1b5b6f445 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c96acf1a2c24951a47b7bf1b5b6f445; no voters:
I20260808 07:06:36.756075 4943 leader_election.cc:288] T 00000000000000000000000000000000 P 7c96acf1a2c24951a47b7bf1b5b6f445 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:36.756191 4953 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c96acf1a2c24951a47b7bf1b5b6f445 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:36.756367 4943 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c96acf1a2c24951a47b7bf1b5b6f445 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:36.756376 4953 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c96acf1a2c24951a47b7bf1b5b6f445 [term 1 LEADER]: Becoming Leader. State: Replica: 7c96acf1a2c24951a47b7bf1b5b6f445, State: Running, Role: LEADER
I20260808 07:06:36.756680 4953 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c96acf1a2c24951a47b7bf1b5b6f445 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c96acf1a2c24951a47b7bf1b5b6f445" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33501 } }
I20260808 07:06:36.757634 4954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c96acf1a2c24951a47b7bf1b5b6f445 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c96acf1a2c24951a47b7bf1b5b6f445" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c96acf1a2c24951a47b7bf1b5b6f445" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33501 } } }
I20260808 07:06:36.757640 4955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c96acf1a2c24951a47b7bf1b5b6f445 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c96acf1a2c24951a47b7bf1b5b6f445. Latest consensus state: current_term: 1 leader_uuid: "7c96acf1a2c24951a47b7bf1b5b6f445" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c96acf1a2c24951a47b7bf1b5b6f445" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33501 } } }
I20260808 07:06:36.757782 4954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c96acf1a2c24951a47b7bf1b5b6f445 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:36.757789 4955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c96acf1a2c24951a47b7bf1b5b6f445 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:36.758147 4962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:36.758718 4962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:36.760383 4962 catalog_manager.cc:1347] Generated new cluster ID: 4bd6a81fc5584bf09696b67932e53aea
I20260808 07:06:36.760448 4962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:36.779196 4962 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:36.779414 4962 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:36.786757 4962 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c96acf1a2c24951a47b7bf1b5b6f445: Generated new TSK 0
I20260808 07:06:36.787035 4962 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:36.795627 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 4871
2026-08-08T07:06:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1464 ms)
[----------] 18 tests from AuthTokenIssuingTest (26696 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:36 dist-test-slave-8tdl krb5kdc[4976](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:36 dist-test-slave-8tdl krb5kdc[4976](info): set up 2 sockets
Aug 08 07:06:36 dist-test-slave-8tdl krb5kdc[4976](info): commencing operation
krb5kdc: starting...
W20260808 07:06:37.956147 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:06:38 dist-test-slave-8tdl krb5kdc[4976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:06:38.107193 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:33453
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:46803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:33453,127.13.207.61:37211,127.13.207.60:45369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:38.203563 4990 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:06:38.203848 4990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:38.203881 4990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:38.203934 4990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:38.205143 4990 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:38.205225 4990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:38.205250 4990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:38.205272 4990 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:38.205303 4990 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:38.207026 4990 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:46803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:33453,127.13.207.61:37211,127.13.207.60:45369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:33453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:38.207386 4990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:38.207652 4990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:06:38.214110 4990 server_base.cc:1034] running on GCE node
W20260808 07:06:38.214069 4995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:38.214156 4996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:38.214084 4998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:38.214741 4990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:38.215165 4990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:38.216372 4990 hybrid_clock.cc:648] HybridClock initialized: now 1786172798216366 us; error 44 us; skew 500 ppm
Aug 08 07:06:38 dist-test-slave-8tdl krb5kdc[4976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172798, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:38.220161 4990 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:06:38.220794 4990 webserver.cc:460] Webserver started at http://127.13.207.62:44559/ using document root <none> and password file <none>
I20260808 07:06:38.221091 4990 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:38.221177 4990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:38.221359 4990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:38.222625 4990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "980adf334c3a4cf8ab6d39ad05e75749"
format_stamp: "Formatted at 2026-08-08 07:06:38 on dist-test-slave-8tdl"
server_key: "1fc2a2a0bb1faecace8aa124eaa24509"
server_key_iv: "7cf540c5601ced1ff34e19b998f795ac"
server_key_version: "encryptionkey@0"
I20260808 07:06:38.223140 4990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "980adf334c3a4cf8ab6d39ad05e75749"
format_stamp: "Formatted at 2026-08-08 07:06:38 on dist-test-slave-8tdl"
server_key: "1fc2a2a0bb1faecace8aa124eaa24509"
server_key_iv: "7cf540c5601ced1ff34e19b998f795ac"
server_key_version: "encryptionkey@0"
I20260808 07:06:38.224817 4990 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:38.225800 5005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:38.226135 4990 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:38.226262 4990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "980adf334c3a4cf8ab6d39ad05e75749"
format_stamp: "Formatted at 2026-08-08 07:06:38 on dist-test-slave-8tdl"
server_key: "1fc2a2a0bb1faecace8aa124eaa24509"
server_key_iv: "7cf540c5601ced1ff34e19b998f795ac"
server_key_version: "encryptionkey@0"
I20260808 07:06:38.226353 4990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:38.263564 4990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:38.266144 4990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:38.266335 4990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:38.271047 4990 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:33453
I20260808 07:06:38.271099 5058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:33453 every 8 connection(s)
I20260808 07:06:38.271517 4990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:38.272552 5059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:38.274288 5059 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33453 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:06:38.274857 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 4990
I20260808 07:06:38.274981 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:38.275264 14140 external_mini_cluster.cc:1428] Setting key 35e8888a913584e0e4a08b0ec0886f23
Aug 08 07:06:38 dist-test-slave-8tdl krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172798, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.207.61@KRBTEST.COM; defaulting to no policy
I20260808 07:06:38.288425 5062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.275402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:33453 (local address 127.0.0.1:33388)
0808 07:06:38.275689 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.275732 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:38.276517 (+ 785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:38.276848 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:38.276851 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:38.277264 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:38.277488 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:38.277493 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.279132 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.279133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:38.279846 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:38.279848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.280634 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.280635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:38.280693 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.281354 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:38.281364 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:38.283694 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:38.286165 (+ 2471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.286174 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.286177 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.286415 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.286844 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.286845 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.286845 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.286959 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.287593 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:38.287601 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:38.287703 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 07:06:38.287861 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 07:06:38.288201 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":166,"thread_start_us":174,"threads_started":1}
I20260808 07:06:38.288425 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.275432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33388 (local address 127.13.207.62:33453)
0808 07:06:38.275692 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.275702 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.275898 (+ 196us) server_negotiation.cc:407] Connection header received
0808 07:06:38.276701 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.276703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.276722 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.276807 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.277629 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.278996 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.279951 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.280513 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.280649 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.283960 (+ 3311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:38.283971 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:38.283972 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:38.283981 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:38.286049 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.286545 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.286579 (+ 34us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.286580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.286750 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.287142 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.287143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.287144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.287497 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:38.287585 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:38.287912 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 07:06:38.288140 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":133,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.61@KRBTEST.COM" created.
I20260808 07:06:38.289762 5059 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:06:38.290271 5062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.290093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:37211 (local address 127.0.0.1:52124)
0808 07:06:38.290154 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.290228 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.61:37211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 07:06:38.290470 5006 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.207.61:37211: connect: Connection refused (error 111)
I20260808 07:06:38.291081 5062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.290959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:37211 (local address 127.0.0.1:52136)
0808 07:06:38.291011 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.291044 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.61:37211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 07:06:38.291285 5059 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.61:37211: Network error: Client connection negotiation failed: client connection to 127.13.207.61:37211: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.61.keytab.
Entry for principal kudu/127.13.207.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.61.keytab.
I20260808 07:06:38.311081 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.61:37211
--webserver_interface=127.13.207.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.61.keytab
--principal=kudu/127.13.207.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:46803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:33453,127.13.207.61:37211,127.13.207.60:45369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:06:38.340597 5059 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } attempt: 1
I20260808 07:06:38.341267 5062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.341100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:37211 (local address 127.0.0.1:52144)
0808 07:06:38.341183 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.341228 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.61:37211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 07:06:38.341706 5062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.341610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:37211 (local address 127.0.0.1:52148)
0808 07:06:38.341646 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.341674 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.61:37211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260808 07:06:38.341815 5059 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.61:37211: Network error: Client connection negotiation failed: client connection to 127.13.207.61:37211: connect: Connection refused (error 111)
W20260808 07:06:38.405723 5066 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:06:38.406001 5066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:38.406020 5066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:38.406045 5066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:38.407415 5066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:38.407472 5066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:38.407487 5066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:38.407500 5066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:38.407516 5066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:38.409240 5066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:46803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:33453,127.13.207.61:37211,127.13.207.60:45369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.61.keytab
--principal=kudu/127.13.207.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.61:37211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.207.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:38.409549 5066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:38.409804 5066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:06:38.409976 5059 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } attempt: 2
I20260808 07:06:38.410493 5062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.410316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:37211 (local address 127.0.0.1:52158)
0808 07:06:38.410394 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.410450 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.61:37211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 07:06:38.410929 5062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.410814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:37211 (local address 127.0.0.1:52174)
0808 07:06:38.410871 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.410905 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.61:37211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 07:06:38.411060 5059 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.61:37211: Network error: Client connection negotiation failed: client connection to 127.13.207.61:37211: connect: Connection refused (error 111)
W20260808 07:06:38.415702 5072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:38.416263 5074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:38.416298 5071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:38.416332 5066 server_base.cc:1034] running on GCE node
I20260808 07:06:38.416744 5066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:38.417146 5066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:38.418336 5066 hybrid_clock.cc:648] HybridClock initialized: now 1786172798418313 us; error 33 us; skew 500 ppm
Aug 08 07:06:38 dist-test-slave-8tdl krb5kdc[4976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172798, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:38.421622 5066 init.cc:377] Logged in from keytab as kudu/127.13.207.61@KRBTEST.COM (short username kudu)
I20260808 07:06:38.422256 5066 webserver.cc:460] Webserver started at http://127.13.207.61:39997/ using document root <none> and password file <none>
I20260808 07:06:38.422580 5066 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:38.422633 5066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:38.422819 5066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:38.424002 5066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-1/data/instance:
uuid: "f839f384157245a6afde14e24fa3d79b"
format_stamp: "Formatted at 2026-08-08 07:06:38 on dist-test-slave-8tdl"
server_key: "dd9a7c2db2a3909d6208c4289705965d"
server_key_iv: "997b1b70b85bd0ce39f8dba868bb0ba2"
server_key_version: "encryptionkey@0"
I20260808 07:06:38.424401 5066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-1/wal/instance:
uuid: "f839f384157245a6afde14e24fa3d79b"
format_stamp: "Formatted at 2026-08-08 07:06:38 on dist-test-slave-8tdl"
server_key: "dd9a7c2db2a3909d6208c4289705965d"
server_key_iv: "997b1b70b85bd0ce39f8dba868bb0ba2"
server_key_version: "encryptionkey@0"
I20260808 07:06:38.425956 5066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:06:38.427223 5081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:38.427472 5066 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:38.427627 5066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-1/wal
uuid: "f839f384157245a6afde14e24fa3d79b"
format_stamp: "Formatted at 2026-08-08 07:06:38 on dist-test-slave-8tdl"
server_key: "dd9a7c2db2a3909d6208c4289705965d"
server_key_iv: "997b1b70b85bd0ce39f8dba868bb0ba2"
server_key_version: "encryptionkey@0"
I20260808 07:06:38.427692 5066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:38.441586 5066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:38.444664 5066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:38.444857 5066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:38.450196 5066 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.61:37211
I20260808 07:06:38.450234 5135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.61:37211 every 8 connection(s)
I20260808 07:06:38.450649 5066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-1/data/info.pb
I20260808 07:06:38.451928 5137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:38.454423 5137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33453 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:06:38.458616 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 5066
I20260808 07:06:38.458807 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-1/wal/instance
I20260808 07:06:38.459023 14140 external_mini_cluster.cc:1428] Setting key f7b056079889bab74822ee02bd2fbc77
Aug 08 07:06:38 dist-test-slave-8tdl krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172798, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.61@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:06:38.464977 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.455512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33398 (local address 127.13.207.62:33453)
0808 07:06:38.455597 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.455600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.455816 (+ 216us) server_negotiation.cc:407] Connection header received
0808 07:06:38.456460 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.456462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.456479 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.456538 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.457253 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.457761 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.458874 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.459155 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.459251 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.461695 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:38.461697 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:38.461697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:38.461713 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:38.463366 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.463817 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.463818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.463818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.463887 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.464236 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.464237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.464237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.464367 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:38.464453 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:38.464750 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 07:06:38.464843 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 07:06:38.465236 5139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.455428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:33453 (local address 127.0.0.1:33398)
0808 07:06:38.455672 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.455723 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:38.456328 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:38.456640 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:38.456644 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:38.456925 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:38.457153 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:38.457157 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.457869 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.457871 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:38.458739 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:38.458742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.459292 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.459294 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:38.459427 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.460024 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:38.460032 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:38.461565 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:38.463465 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.463475 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.463478 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.463731 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.464040 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.464041 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.464041 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.464163 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.464444 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:38.464452 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:38.464606 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 07:06:38.464718 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 07:06:38.464936 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":126,"threads_started":1}
I20260808 07:06:38.465981 5137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.207.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.60@KRBTEST.COM" created.
Aug 08 07:06:38 dist-test-slave-8tdl krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172798, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.61@KRBTEST.COM for kudu/127.13.207.61@KRBTEST.COM
I20260808 07:06:38.475083 5141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.466413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52180 (local address 127.13.207.61:37211)
0808 07:06:38.466606 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.466610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.466627 (+ 17us) server_negotiation.cc:407] Connection header received
0808 07:06:38.466644 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.466646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.466656 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.466777 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.467130 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.467927 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.468770 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.469079 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.469174 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.471555 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:38.471566 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:38.471566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:38.471575 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:38.473492 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.474000 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.474003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.474004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.474063 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.474365 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.474366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.474366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.474543 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:38.474623 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:38.474904 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 07:06:38.474981 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":107,"threads_started":1}
I20260808 07:06:38.475102 5139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.466322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:37211 (local address 127.0.0.1:52180)
0808 07:06:38.466402 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.466413 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:38.466452 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:38.466776 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:38.466777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:38.466992 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:38.467045 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:38.467047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.468024 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.468025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:38.468658 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:38.468660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.469173 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.469174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:38.469244 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.469817 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:38.469822 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:38.471464 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:38.473605 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.473606 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.473606 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.473856 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.474141 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.474142 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.474142 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.474219 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.474629 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:38.474630 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:38.474713 (+ 83us) client_negotiation.cc:769] Sending connection context
0808 07:06:38.474796 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 07:06:38.474927 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 07:06:38.475950 5137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:06:38.476359 5139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.476210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:45369 (local address 127.0.0.1:45408)
0808 07:06:38.476275 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.476335 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:45369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 07:06:38.476522 5085 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.207.60:45369: connect: Connection refused (error 111)
I20260808 07:06:38.477021 5139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.476925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:45369 (local address 127.0.0.1:45422)
0808 07:06:38.476973 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.476999 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:45369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 07:06:38.477145 5137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.60:45369: Network error: Client connection negotiation failed: client connection to 127.13.207.60:45369: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.60.keytab.
Entry for principal kudu/127.13.207.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.60.keytab.
I20260808 07:06:38.496029 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.60:45369
--webserver_interface=127.13.207.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.60.keytab
--principal=kudu/127.13.207.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:46803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:33453,127.13.207.61:37211,127.13.207.60:45369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:06:38.502260 5059 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } attempt: 3
Aug 08 07:06:38 dist-test-slave-8tdl krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172798, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for kudu/127.13.207.61@KRBTEST.COM
I20260808 07:06:38.511713 5062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.502761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:37211 (local address 127.0.0.1:52188)
0808 07:06:38.502834 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.502845 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:38.502917 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:38.503200 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:38.503202 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:38.503387 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:38.503457 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:38.503458 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.504151 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.504152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:38.504677 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:38.504679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.505126 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.505127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:38.505219 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.505842 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:38.505848 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:38.507958 (+ 2110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:38.510391 (+ 2433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.510393 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.510393 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.510650 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.510951 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.510952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.510952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.511025 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.511357 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:38.511358 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:38.511417 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 07:06:38.511482 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 07:06:38.511597 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 07:06:38.511821 5141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.502849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52188 (local address 127.13.207.61:37211)
0808 07:06:38.502933 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.502936 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.502953 (+ 17us) server_negotiation.cc:407] Connection header received
0808 07:06:38.503045 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.503047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.503072 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.503175 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.503548 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.504060 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.504783 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.505035 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.505129 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.508202 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:38.508204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:38.508204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:38.508213 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:38.510259 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.510794 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.510799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.510800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.510867 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.511151 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.511152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.511153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.511289 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:38.511349 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:38.511625 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 07:06:38.511709 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 07:06:38.512390 5059 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:06:38.512820 5062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.512665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:45369 (local address 127.0.0.1:45428)
0808 07:06:38.512752 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.512793 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:45369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 07:06:38.513217 5062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.513118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:45369 (local address 127.0.0.1:45442)
0808 07:06:38.513166 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.513198 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:45369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 07:06:38.513307 5059 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.60:45369: Network error: Client connection negotiation failed: client connection to 127.13.207.60:45369: connect: Connection refused (error 111)
I20260808 07:06:38.526445 5137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } attempt: 1
I20260808 07:06:38.527141 5139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.526991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:45369 (local address 127.0.0.1:45448)
0808 07:06:38.527068 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.527108 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:45369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 07:06:38.527546 5139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.527443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:45369 (local address 127.0.0.1:45454)
0808 07:06:38.527492 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.527518 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:45369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 07:06:38.527647 5137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.60:45369: Network error: Client connection negotiation failed: client connection to 127.13.207.60:45369: connect: Connection refused (error 111)
I20260808 07:06:38.544481 5059 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } attempt: 1
I20260808 07:06:38.545061 5062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.544852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:45369 (local address 127.0.0.1:45464)
0808 07:06:38.544974 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.545020 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:45369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 07:06:38.545472 5062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.545335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:45369 (local address 127.0.0.1:45480)
0808 07:06:38.545417 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.545444 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:45369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260808 07:06:38.545565 5059 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.60:45369: Network error: Client connection negotiation failed: client connection to 127.13.207.60:45369: connect: Connection refused (error 111)
I20260808 07:06:38.595863 5137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } attempt: 2
I20260808 07:06:38.596580 5139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.596290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:45369 (local address 127.0.0.1:45488)
0808 07:06:38.596482 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.596540 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:45369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 07:06:38.596001 5144 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:06:38.596727 5144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:38.596792 5144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:38.596835 5144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 07:06:38.597112 5139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.596992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:45369 (local address 127.0.0.1:45498)
0808 07:06:38.597049 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.597086 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:45369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 07:06:38.597319 5137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.60:45369: Network error: Client connection negotiation failed: client connection to 127.13.207.60:45369: connect: Connection refused (error 111)
W20260808 07:06:38.598590 5144 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:38.598651 5144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:38.598667 5144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:38.598739 5144 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:38.598758 5144 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:38.600596 5144 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:46803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:33453,127.13.207.61:37211,127.13.207.60:45369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.60.keytab
--principal=kudu/127.13.207.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.60:45369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.207.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:38.600901 5144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:38.601202 5144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:38.607012 5151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:38.607017 5150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:38.607056 5144 server_base.cc:1034] running on GCE node
W20260808 07:06:38.607016 5154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:38.607649 5144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:38.608008 5144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:38.609193 5144 hybrid_clock.cc:648] HybridClock initialized: now 1786172798609163 us; error 40 us; skew 500 ppm
Aug 08 07:06:38 dist-test-slave-8tdl krb5kdc[4976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172798, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:38.613093 5144 init.cc:377] Logged in from keytab as kudu/127.13.207.60@KRBTEST.COM (short username kudu)
I20260808 07:06:38.613561 5144 webserver.cc:460] Webserver started at http://127.13.207.60:33491/ using document root <none> and password file <none>
I20260808 07:06:38.613768 5144 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:38.613812 5144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:38.613906 5144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:38.615124 5144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-2/data/instance:
uuid: "3ed9b3d74f8f498089858310670ffe64"
format_stamp: "Formatted at 2026-08-08 07:06:38 on dist-test-slave-8tdl"
server_key: "2ed71da6fd03229d49b8dcc706046512"
server_key_iv: "358f8f9b2cc1717d512e239b8b56720d"
server_key_version: "encryptionkey@0"
I20260808 07:06:38.615487 5144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-2/wal/instance:
uuid: "3ed9b3d74f8f498089858310670ffe64"
format_stamp: "Formatted at 2026-08-08 07:06:38 on dist-test-slave-8tdl"
server_key: "2ed71da6fd03229d49b8dcc706046512"
server_key_iv: "358f8f9b2cc1717d512e239b8b56720d"
server_key_version: "encryptionkey@0"
I20260808 07:06:38.616909 5144 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:06:38.617771 5161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:38.618016 5144 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:38.618119 5144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-2/wal
uuid: "3ed9b3d74f8f498089858310670ffe64"
format_stamp: "Formatted at 2026-08-08 07:06:38 on dist-test-slave-8tdl"
server_key: "2ed71da6fd03229d49b8dcc706046512"
server_key_iv: "358f8f9b2cc1717d512e239b8b56720d"
server_key_version: "encryptionkey@0"
I20260808 07:06:38.618220 5144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:38.620754 5059 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } attempt: 2
I20260808 07:06:38.621253 5062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.621103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:45369 (local address 127.0.0.1:45504)
0808 07:06:38.621170 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.621214 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:45369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 07:06:38.621768 5062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.621646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:45369 (local address 127.0.0.1:45506)
0808 07:06:38.621723 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.621751 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:45369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260808 07:06:38.621888 5059 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.60:45369: Network error: Client connection negotiation failed: client connection to 127.13.207.60:45369: connect: Connection refused (error 111)
I20260808 07:06:38.633961 5144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:38.636656 5144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:38.636832 5144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:38.641333 5144 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.60:45369
I20260808 07:06:38.641407 5213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.60:45369 every 8 connection(s)
I20260808 07:06:38.641776 5144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-2/data/info.pb
I20260808 07:06:38.642952 5214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:38.643293 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 5144
I20260808 07:06:38.643395 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172493058068-14140-0/minicluster-data/master-2/wal/instance
I20260808 07:06:38.643586 14140 external_mini_cluster.cc:1428] Setting key 04fd378cd72908b76392f6ed2c2e4f38
I20260808 07:06:38.645135 5214 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33453 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:06:38 dist-test-slave-8tdl krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.60@KRBTEST.COM
Aug 08 07:06:38 dist-test-slave-8tdl krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172798, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.60@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:06:38.655593 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.646251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33412 (local address 127.13.207.62:33453)
0808 07:06:38.646310 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.646312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.646447 (+ 135us) server_negotiation.cc:407] Connection header received
0808 07:06:38.646528 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.646529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.646541 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.646622 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.647223 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.647963 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.649252 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.649565 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.649658 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.652451 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:38.652453 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:38.652453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:38.652463 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:38.654160 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.654651 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.654652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.654652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.654754 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.654978 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.654979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.654979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.655107 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:38.655188 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:38.655397 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 07:06:38.655490 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 07:06:38.655658 5218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.644229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45518 (local address 127.13.207.60:45369)
0808 07:06:38.644431 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.644439 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.644466 (+ 27us) server_negotiation.cc:407] Connection header received
0808 07:06:38.644910 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.644921 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.644970 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.645096 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.645956 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.647076 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.648172 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.648831 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.649027 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.652046 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:38.652062 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:38.652062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:38.652070 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:38.653941 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.654334 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.654338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.654338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.654430 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.654654 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.654655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.654655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.654958 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:38.655125 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:38.655253 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 07:06:38.655395 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":98,"threads_started":1}
I20260808 07:06:38.655668 5219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.646140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:33453 (local address 127.0.0.1:33412)
0808 07:06:38.646317 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.646375 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:38.646471 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:38.646649 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:38.646652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:38.646976 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:38.647070 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:38.647075 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.648084 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.648085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:38.649152 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:38.649155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.649715 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.649717 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:38.649801 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.650428 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:38.650437 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:38.652344 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:38.654282 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.654291 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.654294 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.654560 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.654835 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.654837 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.654837 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.654910 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.655179 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:38.655186 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:38.655259 (+ 73us) client_negotiation.cc:769] Sending connection context
0808 07:06:38.655352 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 07:06:38.655490 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
I20260808 07:06:38.656455 5214 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:06:38 dist-test-slave-8tdl krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172798, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.60@KRBTEST.COM for kudu/127.13.207.61@KRBTEST.COM
Aug 08 07:06:38 dist-test-slave-8tdl krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:06:38.664726 5219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.656735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:37211 (local address 127.0.0.1:52194)
0808 07:06:38.656789 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.656799 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:38.656833 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:38.657010 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:38.657010 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:38.657129 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:38.657169 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:38.657170 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.658121 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.658122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:38.658774 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:38.658777 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.659207 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.659208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:38.659270 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.659847 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:38.659852 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:38.661289 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:38.663170 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.663171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.663172 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.663497 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.663709 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.663710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.663711 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.663780 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.664372 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:38.664373 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:38.664428 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 07:06:38.664507 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 07:06:38.664601 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 07:06:38.664758 5141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.656823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52194 (local address 127.13.207.61:37211)
0808 07:06:38.656894 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.656896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.656905 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:06:38.656930 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.656931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.656942 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.657001 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.657243 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.658032 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.658875 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.659113 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.659207 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.661386 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:38.661388 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:38.661388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:38.661397 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:38.663065 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.663581 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.663582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.663582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.663639 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.663863 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.663864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.663864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.664002 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:38.664075 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:38.664558 (+ 483us) server_negotiation.cc:299] Negotiation successful
0808 07:06:38.664653 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 07:06:38.665194 5214 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:06:38.665365 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.656627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33416 (local address 127.13.207.62:33453)
0808 07:06:38.656665 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.656667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.656676 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:06:38.656722 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.656722 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.656733 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.657469 (+ 736us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.657505 (+ 36us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.658157 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.658937 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.659152 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.659227 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.662002 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:38.662004 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:38.662004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:38.662012 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:38.663999 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.664358 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.664360 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.664360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.664424 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.664684 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.664684 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.664685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.664968 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:38.665148 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:38.665184 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 07:06:38.665264 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 08 07:06:38 dist-test-slave-8tdl krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172798, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.60@KRBTEST.COM for kudu/127.13.207.60@KRBTEST.COM
Aug 08 07:06:38 dist-test-slave-8tdl krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.61@KRBTEST.COM
I20260808 07:06:38.673467 5218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.665924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45530 (local address 127.13.207.60:45369)
0808 07:06:38.666046 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.666048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.666055 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:06:38.666074 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.666075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.666084 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.666161 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.666449 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.667232 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.668108 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.668498 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.668596 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.670558 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:38.670560 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:38.670560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:38.670567 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:38.672221 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.672618 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.672619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.672619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.672677 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.672880 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.672881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.672881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.673011 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:38.673081 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:38.673230 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 07:06:38.673341 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":11,"server-negotiator.queue_time_us":100}
I20260808 07:06:38.673529 5219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.665610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:45369 (local address 127.0.0.1:45530)
0808 07:06:38.665646 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.665657 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:38.665699 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:38.666163 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:38.666164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:38.666324 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:38.666364 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:38.666367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.667304 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.667306 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:38.668026 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:38.668029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.668607 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.668608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:38.668656 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.669222 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:38.669228 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:38.670447 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:38.672319 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.672320 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.672320 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.672531 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.672754 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.672754 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.672755 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.672818 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.673082 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:38.673083 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:38.673122 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 07:06:38.673207 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 07:06:38.673409 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":93}
I20260808 07:06:38.674451 5141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.666343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52198 (local address 127.13.207.61:37211)
0808 07:06:38.666382 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.666383 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.666410 (+ 27us) server_negotiation.cc:407] Connection header received
0808 07:06:38.666431 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.666431 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.666442 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.666498 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.667102 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.667742 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.668752 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.669044 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.669161 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.671582 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:38.671583 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:38.671584 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:38.671592 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:38.673115 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.673496 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.673497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.673498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.673575 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.673809 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.673810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.673810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.673935 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:38.674086 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:38.674317 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 07:06:38.674388 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:06:38.675464 5214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed9b3d74f8f498089858310670ffe64: Bootstrap starting.
I20260808 07:06:38.676098 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:38.676405 5214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ed9b3d74f8f498089858310670ffe64: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:38.676726 5214 log.cc:824] T 00000000000000000000000000000000 P 3ed9b3d74f8f498089858310670ffe64: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:38.677670 5214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed9b3d74f8f498089858310670ffe64: No bootstrap required, opened a new log
I20260808 07:06:38.677222 5033 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "980adf334c3a4cf8ab6d39ad05e75749"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33416
I20260808 07:06:38.679899 5214 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ed9b3d74f8f498089858310670ffe64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980adf334c3a4cf8ab6d39ad05e75749" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33453 } } peers { permanent_uuid: "f839f384157245a6afde14e24fa3d79b" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } } peers { permanent_uuid: "3ed9b3d74f8f498089858310670ffe64" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } }
I20260808 07:06:38.680092 5214 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ed9b3d74f8f498089858310670ffe64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed9b3d74f8f498089858310670ffe64, State: Initialized, Role: FOLLOWER
I20260808 07:06:38.680181 5214 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ed9b3d74f8f498089858310670ffe64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980adf334c3a4cf8ab6d39ad05e75749" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33453 } } peers { permanent_uuid: "f839f384157245a6afde14e24fa3d79b" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } } peers { permanent_uuid: "3ed9b3d74f8f498089858310670ffe64" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } }
I20260808 07:06:38.680565 5214 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ed9b3d74f8f498089858310670ffe64 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:38.680552 5221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed9b3d74f8f498089858310670ffe64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980adf334c3a4cf8ab6d39ad05e75749" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33453 } } peers { permanent_uuid: "f839f384157245a6afde14e24fa3d79b" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } } peers { permanent_uuid: "3ed9b3d74f8f498089858310670ffe64" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } } }
I20260808 07:06:38.680694 5221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed9b3d74f8f498089858310670ffe64 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:06:38.683058 5235 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ed9b3d74f8f498089858310670ffe64: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:06:38.683137 5235 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:06:38.683362 5144 master_runner.cc:186] Error getting master registration for 127.13.207.62:33453: OK, instance_id { permanent_uuid: "980adf334c3a4cf8ab6d39ad05e75749" instance_seqno: 1786172798269642 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:06:38.686715 5144 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:06:38.688525 5137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } attempt: 3
Aug 08 07:06:38 dist-test-slave-8tdl krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172798, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.61@KRBTEST.COM for kudu/127.13.207.60@KRBTEST.COM
I20260808 07:06:38.697630 5218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.689136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45540 (local address 127.13.207.60:45369)
0808 07:06:38.689202 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.689204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.689213 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:06:38.689261 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.689262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.689271 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.689351 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.689747 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.690484 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.691561 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.691870 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.691941 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.694737 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:38.694739 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:38.694739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:38.694748 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:38.696321 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.696751 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.696752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.696753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.696814 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.697031 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.697032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.697033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.697198 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:38.697279 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:38.697436 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 07:06:38.697526 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:06:38.697630 5139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.689046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:45369 (local address 127.0.0.1:45540)
0808 07:06:38.689112 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.689130 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:38.689198 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:38.689370 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:38.689371 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:38.689594 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:38.689653 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:38.689655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.690587 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.690588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:38.691452 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:38.691456 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.691958 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.691960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:38.692037 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.692713 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:38.692720 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:38.694589 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:38.696447 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.696449 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.696449 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.696662 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.696885 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.696886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.696886 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.696958 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.697270 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:38.697271 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:38.697323 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 07:06:38.697409 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 07:06:38.697526 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 07:06:38.699877 5137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f839f384157245a6afde14e24fa3d79b: Bootstrap starting.
I20260808 07:06:38.700740 5137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f839f384157245a6afde14e24fa3d79b: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:38.701061 5137 log.cc:824] T 00000000000000000000000000000000 P f839f384157245a6afde14e24fa3d79b: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:38.701853 5137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f839f384157245a6afde14e24fa3d79b: No bootstrap required, opened a new log
I20260808 07:06:38.703471 5137 raft_consensus.cc:348] T 00000000000000000000000000000000 P f839f384157245a6afde14e24fa3d79b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980adf334c3a4cf8ab6d39ad05e75749" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33453 } } peers { permanent_uuid: "f839f384157245a6afde14e24fa3d79b" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } } peers { permanent_uuid: "3ed9b3d74f8f498089858310670ffe64" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } }
I20260808 07:06:38.703629 5137 raft_consensus.cc:729] T 00000000000000000000000000000000 P f839f384157245a6afde14e24fa3d79b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f839f384157245a6afde14e24fa3d79b, State: Initialized, Role: FOLLOWER
I20260808 07:06:38.703729 5137 consensus_queue.cc:260] T 00000000000000000000000000000000 P f839f384157245a6afde14e24fa3d79b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980adf334c3a4cf8ab6d39ad05e75749" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33453 } } peers { permanent_uuid: "f839f384157245a6afde14e24fa3d79b" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } } peers { permanent_uuid: "3ed9b3d74f8f498089858310670ffe64" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } }
I20260808 07:06:38.704155 5137 sys_catalog.cc:564] T 00000000000000000000000000000000 P f839f384157245a6afde14e24fa3d79b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:38.704113 5239 sys_catalog.cc:455] T 00000000000000000000000000000000 P f839f384157245a6afde14e24fa3d79b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980adf334c3a4cf8ab6d39ad05e75749" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33453 } } peers { permanent_uuid: "f839f384157245a6afde14e24fa3d79b" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } } peers { permanent_uuid: "3ed9b3d74f8f498089858310670ffe64" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } } }
I20260808 07:06:38.704262 5239 sys_catalog.cc:458] T 00000000000000000000000000000000 P f839f384157245a6afde14e24fa3d79b [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:06:38.706058 5250 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f839f384157245a6afde14e24fa3d79b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:06:38.706146 5250 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:06:38.706408 5066 master_runner.cc:186] Error getting master registration for 127.13.207.62:33453: OK, instance_id { permanent_uuid: "980adf334c3a4cf8ab6d39ad05e75749" instance_seqno: 1786172798269642 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:06:38.709002 5066 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:06:38.721042 5059 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } attempt: 3
Aug 08 07:06:38 dist-test-slave-8tdl krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172798, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for kudu/127.13.207.60@KRBTEST.COM
I20260808 07:06:38.730340 5062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.721477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:45369 (local address 127.0.0.1:45544)
0808 07:06:38.721534 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.721545 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:38.721589 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:38.721829 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:38.721830 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:38.722040 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:38.722100 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:38.722102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.723154 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.723156 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:38.724019 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:38.724022 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.724554 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.724555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:38.724614 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.725169 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:38.725175 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:38.726926 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:38.728990 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.728991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.728992 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.729259 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.729564 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.729565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.729565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.729635 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.729950 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:38.729951 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:38.730032 (+ 81us) client_negotiation.cc:769] Sending connection context
0808 07:06:38.730105 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 07:06:38.730227 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 07:06:38.730836 5218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.721568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45544 (local address 127.13.207.60:45369)
0808 07:06:38.721666 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.721668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.721679 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:06:38.721710 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.721711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.721722 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.721809 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.722206 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.723068 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.724133 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.724462 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.724547 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.727031 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:38.727033 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:38.727034 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:38.727043 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:38.728860 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.729368 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.729370 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.729370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.729442 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.729708 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.729709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.729709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.729861 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:38.729923 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:38.730614 (+ 691us) server_negotiation.cc:299] Negotiation successful
0808 07:06:38.730735 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 07:06:38.732942 5059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749: Bootstrap starting.
I20260808 07:06:38.733775 5059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:38.734091 5059 log.cc:824] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:38.735134 5059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749: No bootstrap required, opened a new log
I20260808 07:06:38.736953 5059 raft_consensus.cc:348] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980adf334c3a4cf8ab6d39ad05e75749" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33453 } } peers { permanent_uuid: "f839f384157245a6afde14e24fa3d79b" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } } peers { permanent_uuid: "3ed9b3d74f8f498089858310670ffe64" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } }
I20260808 07:06:38.737183 5059 raft_consensus.cc:729] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 980adf334c3a4cf8ab6d39ad05e75749, State: Initialized, Role: FOLLOWER
I20260808 07:06:38.737315 5059 consensus_queue.cc:260] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980adf334c3a4cf8ab6d39ad05e75749" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33453 } } peers { permanent_uuid: "f839f384157245a6afde14e24fa3d79b" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } } peers { permanent_uuid: "3ed9b3d74f8f498089858310670ffe64" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } }
I20260808 07:06:38.737730 5059 sys_catalog.cc:564] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:38.737846 5033 raft_consensus.cc:482] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 07:06:38.737938 5033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:38.739073 5033 raft_consensus.cc:504] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980adf334c3a4cf8ab6d39ad05e75749" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33453 } } peers { permanent_uuid: "f839f384157245a6afde14e24fa3d79b" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } } peers { permanent_uuid: "3ed9b3d74f8f498089858310670ffe64" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } }
I20260808 07:06:38.739552 5033 leader_election.cc:288] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749 [CANDIDATE]: Term 1 election: Requested vote from peers f839f384157245a6afde14e24fa3d79b (127.13.207.61:37211), 3ed9b3d74f8f498089858310670ffe64 (127.13.207.60:45369)
W20260808 07:06:38.740151 5271 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 07:06:38.740156 5188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "980adf334c3a4cf8ab6d39ad05e75749" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3ed9b3d74f8f498089858310670ffe64"
I20260808 07:06:38.740320 5188 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ed9b3d74f8f498089858310670ffe64 [term 0 FOLLOWER]: Advancing to term 1
W20260808 07:06:38.740278 5271 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:06:38.741358 5188 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3ed9b3d74f8f498089858310670ffe64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 980adf334c3a4cf8ab6d39ad05e75749 in term 1.
I20260808 07:06:38.741760 5109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "980adf334c3a4cf8ab6d39ad05e75749" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f839f384157245a6afde14e24fa3d79b"
I20260808 07:06:38.741914 5109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f839f384157245a6afde14e24fa3d79b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:38.743018 5109 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f839f384157245a6afde14e24fa3d79b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 980adf334c3a4cf8ab6d39ad05e75749 in term 1.
I20260808 07:06:38.743376 5009 leader_election.cc:302] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ed9b3d74f8f498089858310670ffe64, 980adf334c3a4cf8ab6d39ad05e75749; no voters:
I20260808 07:06:38.747718 4990 master_runner.cc:263] No leader master found from master 980adf334c3a4cf8ab6d39ad05e75749
I20260808 07:06:38.747818 4990 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:06:38.747871 5256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980adf334c3a4cf8ab6d39ad05e75749" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33453 } } peers { permanent_uuid: "f839f384157245a6afde14e24fa3d79b" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } } peers { permanent_uuid: "3ed9b3d74f8f498089858310670ffe64" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } } }
I20260808 07:06:38.747953 5256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:06:38.748008 5256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:38.748060 5256 raft_consensus.cc:686] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749 [term 1 LEADER]: Becoming Leader. State: Replica: 980adf334c3a4cf8ab6d39ad05e75749, State: Running, Role: LEADER
I20260808 07:06:38.748145 5256 consensus_queue.cc:237] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980adf334c3a4cf8ab6d39ad05e75749" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33453 } } peers { permanent_uuid: "f839f384157245a6afde14e24fa3d79b" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } } peers { permanent_uuid: "3ed9b3d74f8f498089858310670ffe64" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } }
I20260808 07:06:38.748661 5256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 980adf334c3a4cf8ab6d39ad05e75749. Latest consensus state: current_term: 1 leader_uuid: "980adf334c3a4cf8ab6d39ad05e75749" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980adf334c3a4cf8ab6d39ad05e75749" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33453 } } peers { permanent_uuid: "f839f384157245a6afde14e24fa3d79b" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } } peers { permanent_uuid: "3ed9b3d74f8f498089858310670ffe64" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } } }
I20260808 07:06:38.748770 5256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:38.754928 5279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:38.755196 5279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:38.761636 5188 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3ed9b3d74f8f498089858310670ffe64 [term 1 FOLLOWER]: Refusing update from remote peer 980adf334c3a4cf8ab6d39ad05e75749: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:38.762004 5256 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ed9b3d74f8f498089858310670ffe64" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:38.763720 5221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed9b3d74f8f498089858310670ffe64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 980adf334c3a4cf8ab6d39ad05e75749. Latest consensus state: current_term: 1 leader_uuid: "980adf334c3a4cf8ab6d39ad05e75749" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980adf334c3a4cf8ab6d39ad05e75749" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33453 } } peers { permanent_uuid: "f839f384157245a6afde14e24fa3d79b" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } } peers { permanent_uuid: "3ed9b3d74f8f498089858310670ffe64" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } } }
I20260808 07:06:38.763827 5221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed9b3d74f8f498089858310670ffe64 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:06:38.764320 5109 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f839f384157245a6afde14e24fa3d79b [term 1 FOLLOWER]: Refusing update from remote peer 980adf334c3a4cf8ab6d39ad05e75749: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:38.764665 5256 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f839f384157245a6afde14e24fa3d79b" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:38.766302 5239 sys_catalog.cc:455] T 00000000000000000000000000000000 P f839f384157245a6afde14e24fa3d79b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 980adf334c3a4cf8ab6d39ad05e75749. Latest consensus state: current_term: 1 leader_uuid: "980adf334c3a4cf8ab6d39ad05e75749" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980adf334c3a4cf8ab6d39ad05e75749" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33453 } } peers { permanent_uuid: "f839f384157245a6afde14e24fa3d79b" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } } peers { permanent_uuid: "3ed9b3d74f8f498089858310670ffe64" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } } }
I20260808 07:06:38.766426 5239 sys_catalog.cc:458] T 00000000000000000000000000000000 P f839f384157245a6afde14e24fa3d79b [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:06:38.767264 5011 tablet.cc:2369] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:06:38.767958 5218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.749182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45554 (local address 127.13.207.60:45369)
0808 07:06:38.749228 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.749230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.756820 (+ 7590us) server_negotiation.cc:407] Connection header received
0808 07:06:38.756941 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.756943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.756954 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.756988 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.757886 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.758730 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.759904 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.760207 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.760248 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.762482 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:38.762483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:38.762484 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:38.762492 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:38.764229 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.766604 (+ 2375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.766607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.766608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.766735 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.767128 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.767129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.767129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.767305 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:38.767391 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:38.767724 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 07:06:38.767834 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:06:38.770629 5273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "980adf334c3a4cf8ab6d39ad05e75749" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980adf334c3a4cf8ab6d39ad05e75749" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33453 } } peers { permanent_uuid: "f839f384157245a6afde14e24fa3d79b" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } } peers { permanent_uuid: "3ed9b3d74f8f498089858310670ffe64" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } } }
I20260808 07:06:38.770823 5273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:38.771368 5239 sys_catalog.cc:455] T 00000000000000000000000000000000 P f839f384157245a6afde14e24fa3d79b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "980adf334c3a4cf8ab6d39ad05e75749" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980adf334c3a4cf8ab6d39ad05e75749" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33453 } } peers { permanent_uuid: "f839f384157245a6afde14e24fa3d79b" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } } peers { permanent_uuid: "3ed9b3d74f8f498089858310670ffe64" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } } }
I20260808 07:06:38.771482 5239 sys_catalog.cc:458] T 00000000000000000000000000000000 P f839f384157245a6afde14e24fa3d79b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:06:38.771723 5283 mvcc.cc:205] Tried to move back new op lower bound from 7316163783725072384 to 7316163783673442304. Current Snapshot: MvccSnapshot[applied={T|T < 7316163783725072384}]
I20260808 07:06:38.771867 5286 mvcc.cc:205] Tried to move back new op lower bound from 7316163783725072384 to 7316163783673442304. Current Snapshot: MvccSnapshot[applied={T|T < 7316163783725072384}]
I20260808 07:06:38.775391 5279 catalog_manager.cc:1347] Generated new cluster ID: ef3f608a67154904a05cb204e1098e8b
I20260808 07:06:38.775482 5279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:38.775627 5256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "980adf334c3a4cf8ab6d39ad05e75749" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980adf334c3a4cf8ab6d39ad05e75749" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33453 } } peers { permanent_uuid: "f839f384157245a6afde14e24fa3d79b" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } } peers { permanent_uuid: "3ed9b3d74f8f498089858310670ffe64" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } } }
I20260808 07:06:38.775779 5256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:38.776675 5221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed9b3d74f8f498089858310670ffe64 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "980adf334c3a4cf8ab6d39ad05e75749" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "980adf334c3a4cf8ab6d39ad05e75749" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 33453 } } peers { permanent_uuid: "f839f384157245a6afde14e24fa3d79b" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 37211 } } peers { permanent_uuid: "3ed9b3d74f8f498089858310670ffe64" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 45369 } } }
I20260808 07:06:38.776918 5221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed9b3d74f8f498089858310670ffe64 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:06:38.780208 5284 mvcc.cc:205] Tried to move back new op lower bound from 7316163783725072384 to 7316163783673442304. Current Snapshot: MvccSnapshot[applied={T|T < 7316163783725072384}]
I20260808 07:06:38.792925 5141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.749965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52200 (local address 127.13.207.61:37211)
0808 07:06:38.750014 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.750016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.767802 (+ 17786us) server_negotiation.cc:407] Connection header received
0808 07:06:38.768750 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.768752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.768765 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.768795 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.772822 (+ 4027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.773650 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.781408 (+ 7758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.781744 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.781804 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.785870 (+ 4066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:38.785872 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:38.785873 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:38.785884 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:38.787721 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.790955 (+ 3234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.790957 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.790958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.791041 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.791596 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.791597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.791597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.791762 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:38.791858 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:38.792700 (+ 842us) server_negotiation.cc:299] Negotiation successful
0808 07:06:38.792820 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:06:38.794333 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.749677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33420 (local address 127.13.207.62:33453)
0808 07:06:38.749727 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.749729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.766738 (+ 17009us) server_negotiation.cc:407] Connection header received
0808 07:06:38.767924 (+ 1186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.767926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.767938 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.767968 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.773802 (+ 5834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.775017 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.780399 (+ 5382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.780715 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.780765 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.788431 (+ 7666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:38.788433 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:38.788434 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:38.788445 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:38.790401 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.791290 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.791292 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.791292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.791356 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.792082 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.792083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.792083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.792236 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:38.792343 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:38.794126 (+ 1783us) server_negotiation.cc:299] Negotiation successful
0808 07:06:38.794231 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:06:38.822360 5279 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:38.823252 5279 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:38.829063 5279 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 980adf334c3a4cf8ab6d39ad05e75749: Generated new TSK 0
I20260808 07:06:38.829314 5279 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:06:38.856866 5218 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:38.855651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45556 (local address 127.13.207.60:45369)
0808 07:06:38.855701 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.855704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.856103 (+ 399us) server_negotiation.cc:407] Connection header received
0808 07:06:38.856219 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.856220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.856232 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.856265 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.856776 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45556: BlockingRecv error: recv got EOF from 127.0.0.1:45556 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260808 07:06:38.858253 5061 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:38.857285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33422 (local address 127.13.207.62:33453)
0808 07:06:38.857326 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.857328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.857649 (+ 321us) server_negotiation.cc:407] Connection header received
0808 07:06:38.857748 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.857749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.857761 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.857791 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.858178 (+ 387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33422: BlockingRecv error: recv got EOF from 127.0.0.1:33422 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 07:06:38.859122 5141 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:38.857529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52202 (local address 127.13.207.61:37211)
0808 07:06:38.857569 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.857571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.858539 (+ 968us) server_negotiation.cc:407] Connection header received
0808 07:06:38.858640 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.858641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.858654 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.858708 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.859048 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52202: BlockingRecv error: recv got EOF from 127.0.0.1:52202 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 07:06:38.861716 5141 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:38.860790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52206 (local address 127.13.207.61:37211)
0808 07:06:38.860842 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.860844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.861121 (+ 277us) server_negotiation.cc:407] Connection header received
0808 07:06:38.861239 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.861240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.861260 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.861291 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.861660 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52206: BlockingRecv error: recv got EOF from 127.0.0.1:52206 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 07:06:38.865298 5141 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:38.864323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52216 (local address 127.13.207.61:37211)
0808 07:06:38.864367 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.864369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.864633 (+ 264us) server_negotiation.cc:407] Connection header received
0808 07:06:38.864745 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.864746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.864759 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.864792 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.865240 (+ 448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52216: BlockingRecv error: recv got EOF from 127.0.0.1:52216 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 07:06:38.869562 5141 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:38.868679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52220 (local address 127.13.207.61:37211)
0808 07:06:38.868722 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.868724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.868978 (+ 254us) server_negotiation.cc:407] Connection header received
0808 07:06:38.869082 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.869083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.869095 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.869125 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.869499 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52220: BlockingRecv error: recv got EOF from 127.0.0.1:52220 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 07:06:38.876325 5141 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:38.875314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52224 (local address 127.13.207.61:37211)
0808 07:06:38.875365 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.875368 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.875657 (+ 289us) server_negotiation.cc:407] Connection header received
0808 07:06:38.875771 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.875773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.875786 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.875818 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.876260 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52224: BlockingRecv error: recv got EOF from 127.0.0.1:52224 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 07:06:38.887161 5141 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:38.885966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52228 (local address 127.13.207.61:37211)
0808 07:06:38.886025 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.886028 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.886341 (+ 313us) server_negotiation.cc:407] Connection header received
0808 07:06:38.886461 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.886463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.886546 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.886586 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.887093 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52228: BlockingRecv error: recv got EOF from 127.0.0.1:52228 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260808 07:06:38.906402 5141 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:38.905374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52234 (local address 127.13.207.61:37211)
0808 07:06:38.905530 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.905533 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.905629 (+ 96us) server_negotiation.cc:407] Connection header received
0808 07:06:38.905771 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.905772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.905788 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.905882 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.906334 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52234: BlockingRecv error: recv got EOF from 127.0.0.1:52234 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 07:06:38.942749 5141 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:38.941384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52240 (local address 127.13.207.61:37211)
0808 07:06:38.941439 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.941442 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.941989 (+ 547us) server_negotiation.cc:407] Connection header received
0808 07:06:38.942164 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.942166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.942181 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.942258 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.942650 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52240: BlockingRecv error: recv got EOF from 127.0.0.1:52240 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 07:06:39.009111 5141 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:39.008188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52248 (local address 127.13.207.61:37211)
0808 07:06:39.008306 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:39.008308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:39.008427 (+ 119us) server_negotiation.cc:407] Connection header received
0808 07:06:39.008468 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:39.008470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:39.008483 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:39.008573 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:39.009057 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52248: BlockingRecv error: recv got EOF from 127.0.0.1:52248 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260808 07:06:39.139734 5141 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:39.138878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52262 (local address 127.13.207.61:37211)
0808 07:06:39.138935 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:39.138938 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:39.139040 (+ 102us) server_negotiation.cc:407] Connection header received
0808 07:06:39.139094 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:39.139095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:39.139108 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:39.139154 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:39.139617 (+ 463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52262: BlockingRecv error: recv got EOF from 127.0.0.1:52262 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 07:06:39.398435 5141 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:39.397529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52264 (local address 127.13.207.61:37211)
0808 07:06:39.397624 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:39.397626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:39.397635 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:06:39.397661 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:39.397662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:39.397673 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:39.397730 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:39.398284 (+ 554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52264: BlockingRecv error: recv got EOF from 127.0.0.1:52264 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 07:06:39.683667 5235 catalog_manager.cc:1259] Loaded cluster ID: ef3f608a67154904a05cb204e1098e8b
I20260808 07:06:39.683794 5235 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3ed9b3d74f8f498089858310670ffe64: loading cluster ID for follower catalog manager: success
I20260808 07:06:39.684643 5235 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3ed9b3d74f8f498089858310670ffe64: acquiring CA information for follower catalog manager: success
I20260808 07:06:39.684962 5235 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3ed9b3d74f8f498089858310670ffe64: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:06:39.706655 5250 catalog_manager.cc:1259] Loaded cluster ID: ef3f608a67154904a05cb204e1098e8b
I20260808 07:06:39.706773 5250 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f839f384157245a6afde14e24fa3d79b: loading cluster ID for follower catalog manager: success
I20260808 07:06:39.707717 5250 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f839f384157245a6afde14e24fa3d79b: acquiring CA information for follower catalog manager: success
I20260808 07:06:39.708091 5250 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f839f384157245a6afde14e24fa3d79b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:06:39.915395 5357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:39.911914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52266 (local address 127.13.207.61:37211)
0808 07:06:39.912191 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:39.912194 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:39.912209 (+ 15us) server_negotiation.cc:407] Connection header received
0808 07:06:39.912336 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:39.912337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:39.912417 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:39.912519 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:39.912725 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:39.913372 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:39.914347 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:39.914573 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:39.914693 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:39.914920 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:39.915092 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:39.915193 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:39.915256 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 07:06:39.915285 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":121,"threads_started":1}
I20260808 07:06:39.920544 5363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:39.917780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45560 (local address 127.13.207.60:45369)
0808 07:06:39.917899 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:39.917901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:39.917911 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:06:39.918000 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:39.918001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:39.918059 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:39.918170 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:39.918303 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:39.918915 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:39.919673 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:39.919895 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:39.919977 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:39.920167 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:39.920282 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:39.920377 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:39.920400 (+ 23us) server_negotiation.cc:299] Negotiation successful
0808 07:06:39.920426 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":59,"threads_started":1}
I20260808 07:06:39.926273 5363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:39.923037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45566 (local address 127.13.207.60:45369)
0808 07:06:39.923143 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:39.923145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:39.923268 (+ 123us) server_negotiation.cc:407] Connection header received
0808 07:06:39.923535 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:39.923536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:39.923550 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:39.923600 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:39.923791 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:39.924547 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:39.925385 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:39.925656 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:39.925735 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:39.925934 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:39.925971 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:39.926043 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:39.926134 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 07:06:39.926165 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 07:06:39.926313 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:39.922886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33434 (local address 127.13.207.62:33453)
0808 07:06:39.923008 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:39.923010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:39.923263 (+ 253us) server_negotiation.cc:407] Connection header received
0808 07:06:39.923363 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:39.923364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:39.923426 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:39.923516 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:39.923617 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:39.924195 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:39.925292 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:39.925639 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:39.925707 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:39.925935 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:39.926078 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:39.926143 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:39.926182 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 07:06:39.926207 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":63,"threads_started":1}
I20260808 07:06:39.926385 5357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:39.923084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52272 (local address 127.13.207.61:37211)
0808 07:06:39.923135 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:39.923137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:39.923269 (+ 132us) server_negotiation.cc:407] Connection header received
0808 07:06:39.923401 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:39.923403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:39.923415 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:39.923479 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:39.923659 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:39.924451 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:39.925510 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:39.925819 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:39.925900 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:39.926099 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:39.926139 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:39.926188 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:39.926277 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 07:06:39.926305 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:06:39.928897 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 4990
I20260808 07:06:39.936296 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 5066
I20260808 07:06:39.944293 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 5144
2026-08-08T07:06:39Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3149 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:40 dist-test-slave-8tdl krb5kdc[5375](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:40 dist-test-slave-8tdl krb5kdc[5375](info): set up 2 sockets
Aug 08 07:06:40 dist-test-slave-8tdl krb5kdc[5375](info): commencing operation
krb5kdc: starting...
W20260808 07:06:41.075594 14140 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/test-user.keytab.
Aug 08 07:06:41 dist-test-slave-8tdl krb5kdc[5375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.207.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
Entry for principal kudu/127.13.207.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab.
I20260808 07:06:41.235471 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:45073
--webserver_interface=127.13.207.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:36041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:45073,127.13.207.61:39749,127.13.207.60:39609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:41.323652 5388 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:06:41.323902 5388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:41.323921 5388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:41.323943 5388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:41.325354 5388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:41.325438 5388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:41.325456 5388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:41.325471 5388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:41.325484 5388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:41.327193 5388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:36041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:45073,127.13.207.61:39749,127.13.207.60:39609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.62.keytab
--principal=kudu/127.13.207.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.62:45073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.207.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:41.327489 5388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:41.327786 5388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:41.333302 5393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:41.333300 5396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:41.333304 5394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:41.334673 5388 server_base.cc:1034] running on GCE node
I20260808 07:06:41.335040 5388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:41.335500 5388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:41.336771 5388 hybrid_clock.cc:648] HybridClock initialized: now 1786172801336738 us; error 48 us; skew 500 ppm
Aug 08 07:06:41 dist-test-slave-8tdl krb5kdc[5375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:41.339946 5388 init.cc:377] Logged in from keytab as kudu/127.13.207.62@KRBTEST.COM (short username kudu)
I20260808 07:06:41.340524 5388 webserver.cc:460] Webserver started at http://127.13.207.62:36957/ using document root <none> and password file <none>
I20260808 07:06:41.340775 5388 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:41.340844 5388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:41.341007 5388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:41.342209 5388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-0/data/instance:
uuid: "062f682eba7a4001826489a8335725d7"
format_stamp: "Formatted at 2026-08-08 07:06:41 on dist-test-slave-8tdl"
server_key: "e04b36a1abdfcaafdd973ccac97c14c4"
server_key_iv: "14e1354f0209b3b46f9cf4d9ff7736b0"
server_key_version: "encryptionkey@0"
I20260808 07:06:41.342602 5388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-0/wal/instance:
uuid: "062f682eba7a4001826489a8335725d7"
format_stamp: "Formatted at 2026-08-08 07:06:41 on dist-test-slave-8tdl"
server_key: "e04b36a1abdfcaafdd973ccac97c14c4"
server_key_iv: "14e1354f0209b3b46f9cf4d9ff7736b0"
server_key_version: "encryptionkey@0"
I20260808 07:06:41.344125 5388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:41.344986 5403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:41.345275 5388 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:41.345424 5388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-0/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-0/wal
uuid: "062f682eba7a4001826489a8335725d7"
format_stamp: "Formatted at 2026-08-08 07:06:41 on dist-test-slave-8tdl"
server_key: "e04b36a1abdfcaafdd973ccac97c14c4"
server_key_iv: "14e1354f0209b3b46f9cf4d9ff7736b0"
server_key_version: "encryptionkey@0"
I20260808 07:06:41.345542 5388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:41.361162 5388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:41.364014 5388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:41.364243 5388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:41.369133 5388 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.62:45073
I20260808 07:06:41.369171 5457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.62:45073 every 8 connection(s)
I20260808 07:06:41.369575 5388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:41.370857 5458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:41.372124 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 5388
I20260808 07:06:41.372272 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-0/wal/instance
I20260808 07:06:41.372515 14140 external_mini_cluster.cc:1428] Setting key ca611c8b81f5e085f7bd16e0e3563eee
I20260808 07:06:41.372941 5458 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.62" port: 45073 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:06:41 dist-test-slave-8tdl krb5kdc[5375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.207.61@KRBTEST.COM; defaulting to no policy
I20260808 07:06:41.386320 5461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.373873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36164 (local address 127.13.207.62:45073)
0808 07:06:41.374659 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.374663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.374756 (+ 93us) server_negotiation.cc:407] Connection header received
0808 07:06:41.375146 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.375148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.375168 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.375267 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.376153 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.377088 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.378510 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.378948 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.379182 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.382534 (+ 3352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:41.382544 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:41.382544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:41.382560 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:41.384520 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.384974 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.384978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.384978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.385073 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.385317 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.385318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.385318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.385582 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:41.385680 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:41.385895 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 07:06:41.386045 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":754,"thread_start_us":123,"threads_started":1}
I20260808 07:06:41.386320 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.373843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:45073 (local address 127.0.0.1:36164)
0808 07:06:41.374066 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.374115 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:41.374923 (+ 808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:41.375320 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:41.375324 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:41.375801 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:41.376028 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:41.376033 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.377250 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.377251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:41.378375 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:41.378379 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.379163 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.379164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:41.379234 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.380029 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:41.380039 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:41.382393 (+ 2354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:41.384637 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:41.384644 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:41.384647 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:41.384873 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:41.385147 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:41.385148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:41.385148 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:41.385247 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:41.385668 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:41.385674 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:41.385742 (+ 68us) client_negotiation.cc:769] Sending connection context
0808 07:06:41.385852 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 07:06:41.386117 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":96,"thread_start_us":123,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.61@KRBTEST.COM" created.
I20260808 07:06:41.387553 5458 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:06:41.388177 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.387918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:39749 (local address 127.0.0.1:44088)
0808 07:06:41.388058 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.388135 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.61:39749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260808 07:06:41.388379 5406 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.207.61:39749: connect: Connection refused (error 111)
I20260808 07:06:41.389117 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.388970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:39749 (local address 127.0.0.1:44092)
0808 07:06:41.389007 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.389091 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.61:39749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260808 07:06:41.389235 5458 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.61:39749: Network error: Client connection negotiation failed: client connection to 127.13.207.61:39749: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.61.keytab.
Entry for principal kudu/127.13.207.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.61.keytab.
I20260808 07:06:41.411543 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.61:39749
--webserver_interface=127.13.207.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.61.keytab
--principal=kudu/127.13.207.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:36041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:45073,127.13.207.61:39749,127.13.207.60:39609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:06:41.438524 5458 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } attempt: 1
I20260808 07:06:41.439152 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.438975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:39749 (local address 127.0.0.1:44108)
0808 07:06:41.439055 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.439106 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.61:39749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 07:06:41.439641 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.439526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:39749 (local address 127.0.0.1:44112)
0808 07:06:41.439582 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.439614 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.61:39749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 07:06:41.439800 5458 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.61:39749: Network error: Client connection negotiation failed: client connection to 127.13.207.61:39749: connect: Connection refused (error 111)
W20260808 07:06:41.504350 5465 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:06:41.504624 5465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:41.504644 5465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:41.504668 5465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:41.505910 5465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:41.505956 5465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:41.505971 5465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:41.505990 5465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:41.506003 5465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:41.507673 5465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:36041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:45073,127.13.207.61:39749,127.13.207.60:39609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.61.keytab
--principal=kudu/127.13.207.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.61:39749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.207.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:41.507944 5465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:41.508033 5458 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } attempt: 2
I20260808 07:06:41.508244 5465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:06:41.508605 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.508442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:39749 (local address 127.0.0.1:44120)
0808 07:06:41.508517 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.508568 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.61:39749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 07:06:41.509130 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.508985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:39749 (local address 127.0.0.1:44128)
0808 07:06:41.509070 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.509102 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.61:39749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260808 07:06:41.509275 5458 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.61:39749: Network error: Client connection negotiation failed: client connection to 127.13.207.61:39749: connect: Connection refused (error 111)
W20260808 07:06:41.513991 5470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:41.514045 5471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:41.514032 5473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:41.514138 5465 server_base.cc:1034] running on GCE node
I20260808 07:06:41.514564 5465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:41.514966 5465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:41.516198 5465 hybrid_clock.cc:648] HybridClock initialized: now 1786172801516182 us; error 32 us; skew 500 ppm
Aug 08 07:06:41 dist-test-slave-8tdl krb5kdc[5375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:41.519486 5465 init.cc:377] Logged in from keytab as kudu/127.13.207.61@KRBTEST.COM (short username kudu)
I20260808 07:06:41.520012 5465 webserver.cc:460] Webserver started at http://127.13.207.61:44263/ using document root <none> and password file <none>
I20260808 07:06:41.520280 5465 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:41.520354 5465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:41.520510 5465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:41.521646 5465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-1/data/instance:
uuid: "56f72bfcace84cecae40dcabf8907c15"
format_stamp: "Formatted at 2026-08-08 07:06:41 on dist-test-slave-8tdl"
server_key: "3e9ca027c8afe484d7730ee9f188bef3"
server_key_iv: "635cc608ce73a6211a31ece38104fa12"
server_key_version: "encryptionkey@0"
I20260808 07:06:41.522032 5465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-1/wal/instance:
uuid: "56f72bfcace84cecae40dcabf8907c15"
format_stamp: "Formatted at 2026-08-08 07:06:41 on dist-test-slave-8tdl"
server_key: "3e9ca027c8afe484d7730ee9f188bef3"
server_key_iv: "635cc608ce73a6211a31ece38104fa12"
server_key_version: "encryptionkey@0"
I20260808 07:06:41.523617 5465 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:06:41.524520 5480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:41.524775 5465 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:41.524893 5465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-1/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-1/wal
uuid: "56f72bfcace84cecae40dcabf8907c15"
format_stamp: "Formatted at 2026-08-08 07:06:41 on dist-test-slave-8tdl"
server_key: "3e9ca027c8afe484d7730ee9f188bef3"
server_key_iv: "635cc608ce73a6211a31ece38104fa12"
server_key_version: "encryptionkey@0"
I20260808 07:06:41.525003 5465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:41.550120 5465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:41.552934 5465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:41.553277 5465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:41.557988 5465 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.61:39749
I20260808 07:06:41.558065 5534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.61:39749 every 8 connection(s)
I20260808 07:06:41.558467 5465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-1/data/info.pb
I20260808 07:06:41.559114 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 5465
I20260808 07:06:41.559238 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-1/wal/instance
I20260808 07:06:41.559448 14140 external_mini_cluster.cc:1428] Setting key 14b68a0de285ceaefd5924c3dba294d9
I20260808 07:06:41.559651 5535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:41.561635 5535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.62" port: 45073 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.207.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.207.60@KRBTEST.COM" created.
Aug 08 07:06:41 dist-test-slave-8tdl krb5kdc[5375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.61@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:06:41.577289 5461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.562602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36172 (local address 127.13.207.62:45073)
0808 07:06:41.562670 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.562673 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.562926 (+ 253us) server_negotiation.cc:407] Connection header received
0808 07:06:41.563616 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.563617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.563631 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.563705 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.564489 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.565007 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.566152 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.566416 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.566514 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.572650 (+ 6136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:41.572652 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:41.572652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:41.572661 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:41.575234 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.575833 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.575835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.575836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.575917 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.576343 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.576344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.576345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.576539 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:41.576631 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:41.577024 (+ 393us) server_negotiation.cc:299] Negotiation successful
0808 07:06:41.577147 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:06:41.577436 5539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.562548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:45073 (local address 127.0.0.1:36172)
0808 07:06:41.562792 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.562833 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:41.563493 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:41.563799 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:41.563803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:41.564157 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:41.564379 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:41.564384 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.565149 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.565150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:41.566047 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:41.566049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.566543 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.566545 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:41.566729 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.567373 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:41.567382 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:41.572421 (+ 5039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:41.575405 (+ 2984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:41.575419 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:41.575422 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:41.575732 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:41.576077 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:41.576078 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:41.576079 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:41.576253 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:41.576635 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:41.576645 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:41.576801 (+ 156us) client_negotiation.cc:769] Sending connection context
0808 07:06:41.576970 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 07:06:41.577213 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":137,"threads_started":1}
I20260808 07:06:41.578477 5535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:06:41 dist-test-slave-8tdl krb5kdc[5375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.61@KRBTEST.COM for kudu/127.13.207.61@KRBTEST.COM
I20260808 07:06:41.590561 5539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.578971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:39749 (local address 127.0.0.1:44144)
0808 07:06:41.579037 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.579050 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:41.579110 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:41.579475 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:41.579476 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:41.579661 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:41.579725 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:41.579727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.580669 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.580670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:41.581202 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:41.581204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.581746 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.581747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:41.581844 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.582365 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:41.582371 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:41.587053 (+ 4682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:41.589264 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:41.589265 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:41.589265 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:41.589456 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:41.589756 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:41.589757 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:41.589758 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:41.589821 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:41.590157 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:41.590158 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:41.590199 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 07:06:41.590286 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 07:06:41.590432 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 07:06:41.590667 5541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.579047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44144 (local address 127.13.207.61:39749)
0808 07:06:41.579282 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.579286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.579309 (+ 23us) server_negotiation.cc:407] Connection header received
0808 07:06:41.579337 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.579339 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.579352 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.579473 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.579820 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.580565 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.581299 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.581649 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.581747 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.587190 (+ 5443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:41.587209 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:41.587209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:41.587219 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:41.589122 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.589612 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.589616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.589616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.589679 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.589891 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.589892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.589892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.590076 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:41.590172 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:41.590323 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 07:06:41.590531 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":97,"server-negotiator.queue_time_us":200,"thread_start_us":128,"threads_started":1}
I20260808 07:06:41.591502 5535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:06:41.591986 5539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.591830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:39609 (local address 127.0.0.1:47884)
0808 07:06:41.591891 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.591955 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:39609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 07:06:41.592157 5482 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.207.60:39609: connect: Connection refused (error 111)
I20260808 07:06:41.592666 5539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.592566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:39609 (local address 127.0.0.1:47894)
0808 07:06:41.592606 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.592640 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:39609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 07:06:41.592839 5535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.60:39609: Network error: Client connection negotiation failed: client connection to 127.13.207.60:39609: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.207.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.60.keytab.
Entry for principal kudu/127.13.207.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.60.keytab.
I20260808 07:06:41.595763 14140 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzjmV4g/build/release/bin/kudu
/tmp/dist-test-taskzjmV4g/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.60:39609
--webserver_interface=127.13.207.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.60.keytab
--principal=kudu/127.13.207.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.207.20:36041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.207.62:45073,127.13.207.61:39749,127.13.207.60:39609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:06:41.600456 5458 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } attempt: 3
Aug 08 07:06:41 dist-test-slave-8tdl krb5kdc[5375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for kudu/127.13.207.61@KRBTEST.COM
I20260808 07:06:41.609086 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.600854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:39749 (local address 127.0.0.1:44158)
0808 07:06:41.600926 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.600942 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:41.601020 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:41.601180 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:41.601181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:41.601383 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:41.601452 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:41.601454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.602209 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.602211 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:41.602939 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:41.602942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.603450 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.603451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:41.603542 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.604056 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:41.604062 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:41.605669 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:41.607794 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:41.607795 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:41.607795 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:41.608061 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:41.608313 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:41.608314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:41.608315 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:41.608399 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:41.608715 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:41.608716 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:41.608765 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 07:06:41.608898 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 07:06:41.609004 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 07:06:41.609153 5541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.600952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44158 (local address 127.13.207.61:39749)
0808 07:06:41.601029 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.601032 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.601047 (+ 15us) server_negotiation.cc:407] Connection header received
0808 07:06:41.601082 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.601083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.601094 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.601175 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.601544 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.602106 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.603076 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.603340 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.603450 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.605778 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:41.605780 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:41.605781 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:41.605790 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:41.607683 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.608149 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.608150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.608151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.608217 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.608476 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.608477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.608477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.608625 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:41.608695 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:41.608931 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 07:06:41.609021 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 07:06:41.609634 5458 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:06:41.610049 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.609928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:39609 (local address 127.0.0.1:47904)
0808 07:06:41.609986 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.610023 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:39609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 07:06:41.610417 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.610304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:39609 (local address 127.0.0.1:47916)
0808 07:06:41.610355 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.610387 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:39609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 07:06:41.610515 5458 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.60:39609: Network error: Client connection negotiation failed: client connection to 127.13.207.60:39609: connect: Connection refused (error 111)
I20260808 07:06:41.641662 5458 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } attempt: 1
I20260808 07:06:41.642087 5535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } attempt: 1
I20260808 07:06:41.642259 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.642100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:39609 (local address 127.0.0.1:47926)
0808 07:06:41.642172 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.642221 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:39609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 07:06:41.642526 5539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.642389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:39609 (local address 127.0.0.1:47936)
0808 07:06:41.642458 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.642496 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:39609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 07:06:41.642956 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.642716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:39609 (local address 127.0.0.1:47946)
0808 07:06:41.642896 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.642928 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:39609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162}
W20260808 07:06:41.643072 5458 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.60:39609: Network error: Client connection negotiation failed: client connection to 127.13.207.60:39609: connect: Connection refused (error 111)
I20260808 07:06:41.643144 5539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.642829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:39609 (local address 127.0.0.1:47954)
0808 07:06:41.642906 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.643111 (+ 205us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:39609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 07:06:41.643285 5535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.60:39609: Network error: Client connection negotiation failed: client connection to 127.13.207.60:39609: connect: Connection refused (error 111)
W20260808 07:06:41.688850 5543 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:06:41.689122 5543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:41.689142 5543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:41.689165 5543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:41.690486 5543 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:41.690550 5543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:41.690565 5543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:41.690578 5543 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:41.690591 5543 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:41.693058 5543 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.207.20:36041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.207.62:45073,127.13.207.61:39749,127.13.207.60:39609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/krb5kdc/kudu_127.13.207.60.keytab
--principal=kudu/127.13.207.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.207.60:39609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.207.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:11:30 UTC on ccycloud.quasar-iqclcm.root.comops.site
build id 4574014
I20260808 07:06:41.693424 5543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:41.693733 5543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:06:41.700111 5543 server_base.cc:1034] running on GCE node
W20260808 07:06:41.700068 5549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:41.700057 5551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:41.700089 5548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:41.700769 5543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:41.701164 5543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:41.702356 5543 hybrid_clock.cc:648] HybridClock initialized: now 1786172801702347 us; error 52 us; skew 500 ppm
Aug 08 07:06:41 dist-test-slave-8tdl krb5kdc[5375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:41.705351 5543 init.cc:377] Logged in from keytab as kudu/127.13.207.60@KRBTEST.COM (short username kudu)
I20260808 07:06:41.705888 5543 webserver.cc:460] Webserver started at http://127.13.207.60:39957/ using document root <none> and password file <none>
I20260808 07:06:41.706171 5543 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:41.706244 5543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:41.706403 5543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:41.707690 5543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-2/data/instance:
uuid: "13c3252dff544e148076501b417c48fd"
format_stamp: "Formatted at 2026-08-08 07:06:41 on dist-test-slave-8tdl"
server_key: "b610cfcfb096245292421cb40fb3169e"
server_key_iv: "b2cfabb8497f75fca3e35cbbfebf9b26"
server_key_version: "encryptionkey@0"
I20260808 07:06:41.708097 5543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-2/wal/instance:
uuid: "13c3252dff544e148076501b417c48fd"
format_stamp: "Formatted at 2026-08-08 07:06:41 on dist-test-slave-8tdl"
server_key: "b610cfcfb096245292421cb40fb3169e"
server_key_iv: "b2cfabb8497f75fca3e35cbbfebf9b26"
server_key_version: "encryptionkey@0"
I20260808 07:06:41.709672 5543 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:06:41.710803 5558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:41.711115 5543 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:41.711233 5543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-2/data,/tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-2/wal
uuid: "13c3252dff544e148076501b417c48fd"
format_stamp: "Formatted at 2026-08-08 07:06:41 on dist-test-slave-8tdl"
server_key: "b610cfcfb096245292421cb40fb3169e"
server_key_iv: "b2cfabb8497f75fca3e35cbbfebf9b26"
server_key_version: "encryptionkey@0"
I20260808 07:06:41.711329 5543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:41.711417 5535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } attempt: 2
I20260808 07:06:41.711875 5539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.711708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:39609 (local address 127.0.0.1:47964)
0808 07:06:41.711791 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.711845 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:39609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 07:06:41.712321 5539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.712200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:39609 (local address 127.0.0.1:47980)
0808 07:06:41.712256 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.712292 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:39609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 07:06:41.712451 5535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.60:39609: Network error: Client connection negotiation failed: client connection to 127.13.207.60:39609: connect: Connection refused (error 111)
I20260808 07:06:41.718255 5458 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } attempt: 2
I20260808 07:06:41.718773 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.718610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:39609 (local address 127.0.0.1:47990)
0808 07:06:41.718702 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.718744 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:39609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 07:06:41.719158 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.719028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:39609 (local address 127.0.0.1:48004)
0808 07:06:41.719114 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.719138 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.207.60:39609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260808 07:06:41.719274 5458 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.207.60:39609: Network error: Client connection negotiation failed: client connection to 127.13.207.60:39609: connect: Connection refused (error 111)
I20260808 07:06:41.733448 5543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:41.736590 5543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:41.736794 5543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:41.742321 5543 rpc_server.cc:307] RPC server started. Bound to: 127.13.207.60:39609
I20260808 07:06:41.742417 5614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.207.60:39609 every 8 connection(s)
I20260808 07:06:41.742938 5543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-2/data/info.pb
I20260808 07:06:41.744674 5615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:41.747364 5615 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.62" port: 45073 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:06:41.752708 14140 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzjmV4g/build/release/bin/kudu as pid 5543
I20260808 07:06:41.752835 14140 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzjmV4g/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172493058068-14140-0/minicluster-data/master-2/wal/instance
I20260808 07:06:41.753314 14140 external_mini_cluster.cc:1428] Setting key 9c3ae5e59abc0e78b868369e25993cb4
Aug 08 07:06:41 dist-test-slave-8tdl krb5kdc[5375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.60@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
Aug 08 07:06:41 dist-test-slave-8tdl krb5kdc[5375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.60@KRBTEST.COM
I20260808 07:06:41.760907 5461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.748374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36182 (local address 127.13.207.62:45073)
0808 07:06:41.748434 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.748437 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.748796 (+ 359us) server_negotiation.cc:407] Connection header received
0808 07:06:41.749601 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.749602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.749618 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.749710 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.750795 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.751598 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.753133 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.753461 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.753540 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.756899 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:41.756901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:41.756902 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:41.756913 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:41.759033 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.759634 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.759636 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.759637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.759734 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.760097 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.760099 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.760099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.760273 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:41.760364 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:41.760681 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 07:06:41.760783 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:06:41.761101 5617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.748384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.62:45073 (local address 127.0.0.1:36182)
0808 07:06:41.748618 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.748681 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:41.749406 (+ 725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:41.749793 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:41.749797 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:41.750265 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:41.750623 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:41.750630 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.751719 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.751720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:41.752953 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:41.752963 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.753640 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.753643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:41.753858 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.754603 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:41.754613 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:41.756775 (+ 2162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:41.759143 (+ 2368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:41.759154 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:41.759157 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:41.759547 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:41.759835 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:41.759836 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:41.759836 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:41.760009 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:41.760388 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:41.760393 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:41.760525 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 07:06:41.760620 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 07:06:41.760793 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":96,"threads_started":1}
I20260808 07:06:41.761766 5615 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:06:41.765861 5619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.753784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48016 (local address 127.13.207.60:39609)
0808 07:06:41.753989 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.753995 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.754022 (+ 27us) server_negotiation.cc:407] Connection header received
0808 07:06:41.754055 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.754070 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.754096 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.754203 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.755171 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.756089 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.757041 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.757612 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.757693 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.761425 (+ 3732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:41.761432 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:41.761432 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:41.761439 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:41.763481 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.763948 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.763952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.763952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.764026 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.764344 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.764345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.764346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.764609 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:41.764758 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:41.765340 (+ 582us) server_negotiation.cc:299] Negotiation successful
0808 07:06:41.765722 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":129,"threads_started":1}
Aug 08 07:06:41 dist-test-slave-8tdl krb5kdc[5375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.60@KRBTEST.COM for kudu/127.13.207.61@KRBTEST.COM
I20260808 07:06:41.770447 5541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.762178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44162 (local address 127.13.207.61:39749)
0808 07:06:41.762236 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.762238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.762290 (+ 52us) server_negotiation.cc:407] Connection header received
0808 07:06:41.762376 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.762377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.762386 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.762438 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.762767 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.763254 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.764180 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.764482 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.764562 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.767310 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:41.767312 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:41.767312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:41.767322 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:41.769167 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.769572 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.769574 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.769574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.769643 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.769931 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.769932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.769932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.770049 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:41.770120 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:41.770299 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 07:06:41.770374 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 07:06:41.770548 5617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.762170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.61:39749 (local address 127.0.0.1:44162)
0808 07:06:41.762237 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.762248 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:41.762305 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:41.762444 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:41.762445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:41.762595 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:41.762665 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:41.762667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.763340 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.763342 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:41.764090 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:41.764094 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.764589 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.764590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:41.764697 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.765385 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:41.765390 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:41.767232 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:41.769289 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:41.769296 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:41.769296 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:41.769492 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:41.769719 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:41.769721 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:41.769721 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:41.769836 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:41.770134 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:41.770135 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:41.770192 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 07:06:41.770281 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 07:06:41.770402 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 07:06:41.771040 5615 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:06:41 dist-test-slave-8tdl krb5kdc[5375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.62@KRBTEST.COM
I20260808 07:06:41.775718 5461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.766989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36188 (local address 127.13.207.62:45073)
0808 07:06:41.767034 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.767036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.767045 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:06:41.767107 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.767108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.767119 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.767180 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.767817 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.768401 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.769126 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.769319 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.769408 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.772306 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:41.772308 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:41.772308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:41.772316 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:41.774092 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.774627 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.774628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.774629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.774773 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.775119 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.775120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.775121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.775283 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:41.775444 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:41.775581 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 07:06:41.775649 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Aug 08 07:06:41 dist-test-slave-8tdl krb5kdc[5375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.60@KRBTEST.COM for kudu/127.13.207.60@KRBTEST.COM
I20260808 07:06:41.779810 5617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.771353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:39609 (local address 127.0.0.1:48030)
0808 07:06:41.771443 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.771458 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:41.771515 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:41.771740 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:41.771742 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:41.771979 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:41.772035 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:41.772037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.772890 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.772891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:41.773713 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:41.773717 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.774167 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.774168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:41.774233 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.774657 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:41.774663 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:41.776551 (+ 1888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:41.778536 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:41.778537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:41.778537 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:41.778832 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:41.779089 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:41.779089 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:41.779090 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:41.779159 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:41.779427 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:41.779428 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:41.779472 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 07:06:41.779550 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:06:41.779673 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 07:06:41.779824 5619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.771550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48030 (local address 127.13.207.60:39609)
0808 07:06:41.771591 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.771593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.771602 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:06:41.771624 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.771624 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.771636 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.771700 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.772130 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.772797 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.773864 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.774083 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.774145 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.776634 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:41.776636 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:41.776636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:41.776644 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:41.778414 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.778934 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.778935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.778935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.779005 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.779228 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.779228 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.779229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.779352 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:41.779444 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:41.779580 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 07:06:41.779735 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":73,"server-negotiator.queue_time_us":24}
Aug 08 07:06:41 dist-test-slave-8tdl krb5kdc[5375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.207.61@KRBTEST.COM
I20260808 07:06:41.781905 5615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13c3252dff544e148076501b417c48fd: Bootstrap starting.
I20260808 07:06:41.783092 5615 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13c3252dff544e148076501b417c48fd: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:41.783432 5615 log.cc:824] T 00000000000000000000000000000000 P 13c3252dff544e148076501b417c48fd: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:41.784368 5615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13c3252dff544e148076501b417c48fd: No bootstrap required, opened a new log
I20260808 07:06:41.784960 5541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.776442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44172 (local address 127.13.207.61:39749)
0808 07:06:41.776508 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.776511 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.776519 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:06:41.776544 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.776545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.776557 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.776628 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.777338 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.777817 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.778506 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.778784 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.778878 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.781545 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:41.781548 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:41.781548 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:41.781558 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:41.783646 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.784079 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.784080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.784081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.784153 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.784378 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.784380 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.784380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.784552 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:41.784730 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:41.784771 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 07:06:41.784851 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:06:41.786819 14140 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:41.786804 5615 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13c3252dff544e148076501b417c48fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "062f682eba7a4001826489a8335725d7" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 45073 } } peers { permanent_uuid: "56f72bfcace84cecae40dcabf8907c15" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } } peers { permanent_uuid: "13c3252dff544e148076501b417c48fd" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } }
I20260808 07:06:41.787014 5615 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13c3252dff544e148076501b417c48fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c3252dff544e148076501b417c48fd, State: Initialized, Role: FOLLOWER
I20260808 07:06:41.787125 5615 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13c3252dff544e148076501b417c48fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "062f682eba7a4001826489a8335725d7" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 45073 } } peers { permanent_uuid: "56f72bfcace84cecae40dcabf8907c15" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } } peers { permanent_uuid: "13c3252dff544e148076501b417c48fd" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } }
I20260808 07:06:41.787474 5615 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13c3252dff544e148076501b417c48fd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:41.788038 5432 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "062f682eba7a4001826489a8335725d7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36188
I20260808 07:06:41.789018 5621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13c3252dff544e148076501b417c48fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "062f682eba7a4001826489a8335725d7" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 45073 } } peers { permanent_uuid: "56f72bfcace84cecae40dcabf8907c15" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } } peers { permanent_uuid: "13c3252dff544e148076501b417c48fd" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } } }
I20260808 07:06:41.789635 5621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13c3252dff544e148076501b417c48fd [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:06:41.789485 5633 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 13c3252dff544e148076501b417c48fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:06:41.790151 5633 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:06:41.790464 5543 master_runner.cc:186] Error getting master registration for 127.13.207.62:45073: OK, instance_id { permanent_uuid: "062f682eba7a4001826489a8335725d7" instance_seqno: 1786172801367759 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:06:41.793582 5543 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:06:41.803606 5535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } attempt: 3
Aug 08 07:06:41 dist-test-slave-8tdl krb5kdc[5375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.61@KRBTEST.COM for kudu/127.13.207.60@KRBTEST.COM
I20260808 07:06:41.812760 5539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.804056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:39609 (local address 127.0.0.1:48036)
0808 07:06:41.804124 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.804139 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:41.804208 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:41.804369 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:41.804370 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:41.804589 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:41.804649 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:41.804651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.805624 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.805626 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:41.806507 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:41.806512 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.807077 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.807079 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:41.807143 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.807772 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:41.807777 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:41.809504 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:41.811286 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:41.811288 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:41.811288 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:41.811539 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:41.811776 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:41.811777 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:41.811777 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:41.811851 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:41.812359 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:41.812360 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:41.812431 (+ 71us) client_negotiation.cc:769] Sending connection context
0808 07:06:41.812529 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 07:06:41.812647 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 07:06:41.812763 5619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.804160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48036 (local address 127.13.207.60:39609)
0808 07:06:41.804226 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.804228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.804240 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:06:41.804271 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.804272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.804286 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.804354 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.804734 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.805482 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.806624 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.806974 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.807057 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.809626 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:41.809628 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:41.809628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:41.809636 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:41.811197 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.811627 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.811628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.811628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.811698 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.811959 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.811960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.811960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.812277 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:41.812355 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:41.812559 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 07:06:41.812659 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:06:41.815310 5535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56f72bfcace84cecae40dcabf8907c15: Bootstrap starting.
I20260808 07:06:41.816138 5535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56f72bfcace84cecae40dcabf8907c15: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:41.816505 5535 log.cc:824] T 00000000000000000000000000000000 P 56f72bfcace84cecae40dcabf8907c15: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:41.817472 5535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56f72bfcace84cecae40dcabf8907c15: No bootstrap required, opened a new log
I20260808 07:06:41.818378 5458 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } attempt: 3
I20260808 07:06:41.819708 5535 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56f72bfcace84cecae40dcabf8907c15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "062f682eba7a4001826489a8335725d7" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 45073 } } peers { permanent_uuid: "56f72bfcace84cecae40dcabf8907c15" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } } peers { permanent_uuid: "13c3252dff544e148076501b417c48fd" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } }
I20260808 07:06:41.819906 5535 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56f72bfcace84cecae40dcabf8907c15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56f72bfcace84cecae40dcabf8907c15, State: Initialized, Role: FOLLOWER
I20260808 07:06:41.820014 5535 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56f72bfcace84cecae40dcabf8907c15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "062f682eba7a4001826489a8335725d7" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 45073 } } peers { permanent_uuid: "56f72bfcace84cecae40dcabf8907c15" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } } peers { permanent_uuid: "13c3252dff544e148076501b417c48fd" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } }
I20260808 07:06:41.820375 5535 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56f72bfcace84cecae40dcabf8907c15 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:41.820397 5636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56f72bfcace84cecae40dcabf8907c15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "062f682eba7a4001826489a8335725d7" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 45073 } } peers { permanent_uuid: "56f72bfcace84cecae40dcabf8907c15" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } } peers { permanent_uuid: "13c3252dff544e148076501b417c48fd" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } } }
I20260808 07:06:41.820693 5636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56f72bfcace84cecae40dcabf8907c15 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:06:41.823258 5647 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 56f72bfcace84cecae40dcabf8907c15: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:06:41.823421 5647 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 08 07:06:41 dist-test-slave-8tdl krb5kdc[5375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.207.62@KRBTEST.COM for kudu/127.13.207.60@KRBTEST.COM
I20260808 07:06:41.822561 5465 master_runner.cc:186] Error getting master registration for 127.13.207.62:45073: OK, instance_id { permanent_uuid: "062f682eba7a4001826489a8335725d7" instance_seqno: 1786172801367759 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:06:41.825608 5465 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:06:41.827399 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.818727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.207.60:39609 (local address 127.0.0.1:48052)
0808 07:06:41.818785 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:41.818796 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:41.818838 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:41.819116 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:41.819118 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:41.819284 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:41.819330 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:41.819331 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.820397 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.820398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:41.821237 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:41.821241 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.821778 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.821779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:41.821838 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.822397 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:41.822403 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:41.824041 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:41.826129 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:41.826130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:41.826131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:41.826372 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:41.826619 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:41.826620 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:41.826620 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:41.826734 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:41.827045 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:41.827046 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:41.827106 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 07:06:41.827177 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 07:06:41.827296 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 07:06:41.827467 5619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.818913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48052 (local address 127.13.207.60:39609)
0808 07:06:41.818969 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.818971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.818978 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:06:41.819010 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.819011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.819041 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.819098 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.819414 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.820280 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.821382 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.821685 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.822051 (+ 366us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.824138 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:41.824140 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:41.824140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:41.824150 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:41.826030 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.826465 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.826467 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.826467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.826542 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.826820 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.826821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.826821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.826971 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:41.827043 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:41.827237 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 07:06:41.827360 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:06:41.829707 5458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7: Bootstrap starting.
I20260808 07:06:41.830525 5458 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:41.830888 5458 log.cc:824] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:41.831843 5458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7: No bootstrap required, opened a new log
I20260808 07:06:41.833375 5458 raft_consensus.cc:348] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "062f682eba7a4001826489a8335725d7" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 45073 } } peers { permanent_uuid: "56f72bfcace84cecae40dcabf8907c15" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } } peers { permanent_uuid: "13c3252dff544e148076501b417c48fd" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } }
I20260808 07:06:41.833535 5458 raft_consensus.cc:729] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 062f682eba7a4001826489a8335725d7, State: Initialized, Role: FOLLOWER
I20260808 07:06:41.833621 5458 consensus_queue.cc:260] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "062f682eba7a4001826489a8335725d7" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 45073 } } peers { permanent_uuid: "56f72bfcace84cecae40dcabf8907c15" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } } peers { permanent_uuid: "13c3252dff544e148076501b417c48fd" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } }
I20260808 07:06:41.833966 5458 sys_catalog.cc:564] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:41.833937 5650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "062f682eba7a4001826489a8335725d7" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 45073 } } peers { permanent_uuid: "56f72bfcace84cecae40dcabf8907c15" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } } peers { permanent_uuid: "13c3252dff544e148076501b417c48fd" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } } }
I20260808 07:06:41.834033 5650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:06:41.835562 5432 raft_consensus.cc:482] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 07:06:41.835635 5432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7 [term 0 FOLLOWER]: Advancing to term 1
W20260808 07:06:41.835978 5662 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:06:41.836030 5662 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:06:41.836928 5432 raft_consensus.cc:504] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "062f682eba7a4001826489a8335725d7" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 45073 } } peers { permanent_uuid: "56f72bfcace84cecae40dcabf8907c15" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } } peers { permanent_uuid: "13c3252dff544e148076501b417c48fd" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } }
I20260808 07:06:41.837198 5432 leader_election.cc:288] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7 [CANDIDATE]: Term 1 election: Requested vote from peers 56f72bfcace84cecae40dcabf8907c15 (127.13.207.61:39749), 13c3252dff544e148076501b417c48fd (127.13.207.60:39609)
I20260808 07:06:41.837715 5508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "062f682eba7a4001826489a8335725d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "56f72bfcace84cecae40dcabf8907c15"
I20260808 07:06:41.837838 5508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56f72bfcace84cecae40dcabf8907c15 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:41.839898 5588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "062f682eba7a4001826489a8335725d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "13c3252dff544e148076501b417c48fd"
I20260808 07:06:41.840068 5588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13c3252dff544e148076501b417c48fd [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:41.841042 5588 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 13c3252dff544e148076501b417c48fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 062f682eba7a4001826489a8335725d7 in term 1.
I20260808 07:06:41.841364 5405 leader_election.cc:302] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 062f682eba7a4001826489a8335725d7, 13c3252dff544e148076501b417c48fd; no voters:
I20260808 07:06:41.841529 5650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:41.841696 5650 raft_consensus.cc:686] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7 [term 1 LEADER]: Becoming Leader. State: Replica: 062f682eba7a4001826489a8335725d7, State: Running, Role: LEADER
I20260808 07:06:41.841790 5650 consensus_queue.cc:237] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "062f682eba7a4001826489a8335725d7" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 45073 } } peers { permanent_uuid: "56f72bfcace84cecae40dcabf8907c15" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } } peers { permanent_uuid: "13c3252dff544e148076501b417c48fd" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } }
I20260808 07:06:41.842253 5650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 062f682eba7a4001826489a8335725d7. Latest consensus state: current_term: 1 leader_uuid: "062f682eba7a4001826489a8335725d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "062f682eba7a4001826489a8335725d7" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 45073 } } peers { permanent_uuid: "56f72bfcace84cecae40dcabf8907c15" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } } peers { permanent_uuid: "13c3252dff544e148076501b417c48fd" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } } }
I20260808 07:06:41.842324 5650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:41.843019 5508 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 56f72bfcace84cecae40dcabf8907c15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 062f682eba7a4001826489a8335725d7 in term 1.
I20260808 07:06:41.851648 5670 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:41.851917 5670 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:41.855607 5619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.842992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48064 (local address 127.13.207.60:39609)
0808 07:06:41.843055 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.843057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.846655 (+ 3598us) server_negotiation.cc:407] Connection header received
0808 07:06:41.846795 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.846796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.846807 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.846839 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.847701 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.848507 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.849828 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.850143 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.850194 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.852252 (+ 2058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:41.852254 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:41.852255 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:41.852264 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:41.854071 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.854528 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.854530 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.854530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.854595 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.854938 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.854939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.854939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.855092 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:41.855168 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:41.855400 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 07:06:41.855490 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:06:41.862741 5541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.843434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44188 (local address 127.13.207.61:39749)
0808 07:06:41.843479 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.843482 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.854919 (+ 11437us) server_negotiation.cc:407] Connection header received
0808 07:06:41.855047 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.855048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.855061 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.855093 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.855794 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.856562 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.857644 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.857969 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.858011 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.859720 (+ 1709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:41.859722 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:41.859722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:41.859730 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:41.861347 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.861768 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.861769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.861769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.861834 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.862058 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.862059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.862059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.862200 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:41.862287 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:41.862525 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 07:06:41.862614 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:06:41.864445 5588 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 13c3252dff544e148076501b417c48fd [term 1 FOLLOWER]: Refusing update from remote peer 062f682eba7a4001826489a8335725d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:41.864773 5508 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56f72bfcace84cecae40dcabf8907c15 [term 1 FOLLOWER]: Refusing update from remote peer 062f682eba7a4001826489a8335725d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:41.864930 5461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.843687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36204 (local address 127.13.207.62:45073)
0808 07:06:41.843726 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.843728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.856269 (+ 12541us) server_negotiation.cc:407] Connection header received
0808 07:06:41.856382 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.856383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.856396 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.856429 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.857143 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.857882 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.859221 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.859536 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.859579 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.861443 (+ 1864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:41.861445 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:41.861445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:41.861453 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:41.863151 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.863889 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.863890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.863890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.863959 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.864180 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.864181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.864181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.864314 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:41.864388 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:41.864729 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 07:06:41.864822 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 07:06:41.865219 5409 tablet.cc:2369] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:06:41.865213 5650 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c3252dff544e148076501b417c48fd" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:41.865388 5668 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56f72bfcace84cecae40dcabf8907c15" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:41.867103 5636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56f72bfcace84cecae40dcabf8907c15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 062f682eba7a4001826489a8335725d7. Latest consensus state: current_term: 1 leader_uuid: "062f682eba7a4001826489a8335725d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "062f682eba7a4001826489a8335725d7" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 45073 } } peers { permanent_uuid: "56f72bfcace84cecae40dcabf8907c15" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } } peers { permanent_uuid: "13c3252dff544e148076501b417c48fd" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } } }
I20260808 07:06:41.867223 5636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56f72bfcace84cecae40dcabf8907c15 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:06:41.867600 5621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13c3252dff544e148076501b417c48fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 062f682eba7a4001826489a8335725d7. Latest consensus state: current_term: 1 leader_uuid: "062f682eba7a4001826489a8335725d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "062f682eba7a4001826489a8335725d7" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 45073 } } peers { permanent_uuid: "56f72bfcace84cecae40dcabf8907c15" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } } peers { permanent_uuid: "13c3252dff544e148076501b417c48fd" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } } }
I20260808 07:06:41.867703 5621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13c3252dff544e148076501b417c48fd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:06:41.872354 5668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "062f682eba7a4001826489a8335725d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "062f682eba7a4001826489a8335725d7" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 45073 } } peers { permanent_uuid: "56f72bfcace84cecae40dcabf8907c15" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } } peers { permanent_uuid: "13c3252dff544e148076501b417c48fd" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } } }
I20260808 07:06:41.872493 5668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:41.872653 5636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56f72bfcace84cecae40dcabf8907c15 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "062f682eba7a4001826489a8335725d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "062f682eba7a4001826489a8335725d7" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 45073 } } peers { permanent_uuid: "56f72bfcace84cecae40dcabf8907c15" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } } peers { permanent_uuid: "13c3252dff544e148076501b417c48fd" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } } }
I20260808 07:06:41.872759 5636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56f72bfcace84cecae40dcabf8907c15 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:06:41.872851 5674 mvcc.cc:205] Tried to move back new op lower bound from 7316163796420894720 to 7316163796344750080. Current Snapshot: MvccSnapshot[applied={T|T < 7316163796420894720}]
I20260808 07:06:41.876398 5650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "062f682eba7a4001826489a8335725d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "062f682eba7a4001826489a8335725d7" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 45073 } } peers { permanent_uuid: "56f72bfcace84cecae40dcabf8907c15" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } } peers { permanent_uuid: "13c3252dff544e148076501b417c48fd" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } } }
I20260808 07:06:41.876523 5650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:41.877017 5676 mvcc.cc:205] Tried to move back new op lower bound from 7316163796420894720 to 7316163796344750080. Current Snapshot: MvccSnapshot[applied={T|T < 7316163796420894720}]
I20260808 07:06:41.877159 5621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13c3252dff544e148076501b417c48fd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "062f682eba7a4001826489a8335725d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "062f682eba7a4001826489a8335725d7" member_type: VOTER last_known_addr { host: "127.13.207.62" port: 45073 } } peers { permanent_uuid: "56f72bfcace84cecae40dcabf8907c15" member_type: VOTER last_known_addr { host: "127.13.207.61" port: 39749 } } peers { permanent_uuid: "13c3252dff544e148076501b417c48fd" member_type: VOTER last_known_addr { host: "127.13.207.60" port: 39609 } } }
I20260808 07:06:41.877247 5621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13c3252dff544e148076501b417c48fd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:06:41.877382 5679 mvcc.cc:205] Tried to move back new op lower bound from 7316163796420894720 to 7316163796344750080. Current Snapshot: MvccSnapshot[applied={T|T < 7316163796420894720}]
I20260808 07:06:41.879628 5670 catalog_manager.cc:1347] Generated new cluster ID: 5355e989f6ba4717b3c660379f43554c
I20260808 07:06:41.879688 5670 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:41.898955 5670 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:41.899892 5670 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:41.918824 5670 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 062f682eba7a4001826489a8335725d7: Generated new TSK 0
I20260808 07:06:41.919263 5670 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:06:41.952723 5461 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:41.950909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36210 (local address 127.13.207.62:45073)
0808 07:06:41.950953 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.950955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.951791 (+ 836us) server_negotiation.cc:407] Connection header received
0808 07:06:41.952030 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.952032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.952046 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.952080 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.952636 (+ 556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36210: BlockingRecv error: recv got EOF from 127.0.0.1:36210 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 07:06:41.953684 5541 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:41.950630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44200 (local address 127.13.207.61:39749)
0808 07:06:41.950702 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.950704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.951955 (+ 1251us) server_negotiation.cc:407] Connection header received
0808 07:06:41.953160 (+ 1205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.953162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.953175 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.953206 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.953616 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44200: BlockingRecv error: recv got EOF from 127.0.0.1:44200 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260808 07:06:41.954842 5619 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:41.951467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48072 (local address 127.13.207.60:39609)
0808 07:06:41.951511 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.951513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.954124 (+ 2611us) server_negotiation.cc:407] Connection header received
0808 07:06:41.954310 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.954312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.954325 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.954356 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.954743 (+ 387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48072: BlockingRecv error: recv got EOF from 127.0.0.1:48072 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 07:06:41.957726 5541 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:41.956815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44214 (local address 127.13.207.61:39749)
0808 07:06:41.956860 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.956862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.957125 (+ 263us) server_negotiation.cc:407] Connection header received
0808 07:06:41.957246 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.957247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.957269 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.957301 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.957667 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44214: BlockingRecv error: recv got EOF from 127.0.0.1:44214 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 07:06:41.961323 5541 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:41.960357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44222 (local address 127.13.207.61:39749)
0808 07:06:41.960399 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.960401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.960701 (+ 300us) server_negotiation.cc:407] Connection header received
0808 07:06:41.960813 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.960814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.960827 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.960859 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.961263 (+ 404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44222: BlockingRecv error: recv got EOF from 127.0.0.1:44222 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 07:06:41.966101 5541 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:41.964783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44224 (local address 127.13.207.61:39749)
0808 07:06:41.964828 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.964830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.965322 (+ 492us) server_negotiation.cc:407] Connection header received
0808 07:06:41.965540 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.965541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.965558 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.965603 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.966038 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44224: BlockingRecv error: recv got EOF from 127.0.0.1:44224 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 07:06:41.973735 5541 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:41.972294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44238 (local address 127.13.207.61:39749)
0808 07:06:41.972354 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.972356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.972991 (+ 635us) server_negotiation.cc:407] Connection header received
0808 07:06:41.973116 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.973118 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.973131 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.973164 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.973668 (+ 504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44238: BlockingRecv error: recv got EOF from 127.0.0.1:44238 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260808 07:06:41.984537 5541 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:41.983481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44240 (local address 127.13.207.61:39749)
0808 07:06:41.983548 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.983550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.983880 (+ 330us) server_negotiation.cc:407] Connection header received
0808 07:06:41.983996 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.983998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.984010 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.984044 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.984459 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44240: BlockingRecv error: recv got EOF from 127.0.0.1:44240 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 07:06:42.003367 5541 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:42.002204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44244 (local address 127.13.207.61:39749)
0808 07:06:42.002253 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:42.002256 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:42.002610 (+ 354us) server_negotiation.cc:407] Connection header received
0808 07:06:42.002816 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:42.002817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:42.002831 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:42.002864 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:42.003298 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44244: BlockingRecv error: recv got EOF from 127.0.0.1:44244 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 07:06:42.038168 5541 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:42.037266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44254 (local address 127.13.207.61:39749)
0808 07:06:42.037338 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:42.037340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:42.037493 (+ 153us) server_negotiation.cc:407] Connection header received
0808 07:06:42.037556 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:42.037557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:42.037567 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:42.037624 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:42.038096 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44254: BlockingRecv error: recv got EOF from 127.0.0.1:44254 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260808 07:06:42.104553 5541 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:42.103672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44256 (local address 127.13.207.61:39749)
0808 07:06:42.103777 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:42.103779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:42.103889 (+ 110us) server_negotiation.cc:407] Connection header received
0808 07:06:42.104022 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:42.104023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:42.104037 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:42.104116 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:42.104481 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44256: BlockingRecv error: recv got EOF from 127.0.0.1:44256 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260808 07:06:42.235131 5541 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:42.233979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44272 (local address 127.13.207.61:39749)
0808 07:06:42.234089 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:42.234092 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:42.234238 (+ 146us) server_negotiation.cc:407] Connection header received
0808 07:06:42.234379 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:42.234380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:42.234394 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:42.234429 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:42.235061 (+ 632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44272: BlockingRecv error: recv got EOF from 127.0.0.1:44272 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260808 07:06:42.493456 5541 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:42.492613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44280 (local address 127.13.207.61:39749)
0808 07:06:42.492694 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:42.492696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:42.492711 (+ 15us) server_negotiation.cc:407] Connection header received
0808 07:06:42.492791 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:42.492792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:42.492803 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:42.492911 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:42.493376 (+ 465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44280: BlockingRecv error: recv got EOF from 127.0.0.1:44280 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 07:06:42.790724 5633 catalog_manager.cc:1259] Loaded cluster ID: 5355e989f6ba4717b3c660379f43554c
I20260808 07:06:42.790813 5633 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 13c3252dff544e148076501b417c48fd: loading cluster ID for follower catalog manager: success
I20260808 07:06:42.791677 5633 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 13c3252dff544e148076501b417c48fd: acquiring CA information for follower catalog manager: success
I20260808 07:06:42.792030 5633 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 13c3252dff544e148076501b417c48fd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:06:42.824028 5647 catalog_manager.cc:1259] Loaded cluster ID: 5355e989f6ba4717b3c660379f43554c
I20260808 07:06:42.824122 5647 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 56f72bfcace84cecae40dcabf8907c15: loading cluster ID for follower catalog manager: success
I20260808 07:06:42.825047 5647 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 56f72bfcace84cecae40dcabf8907c15: acquiring CA information for follower catalog manager: success
I20260808 07:06:42.825414 5647 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 56f72bfcace84cecae40dcabf8907c15: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:06:43.010356 5752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.007108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44284 (local address 127.13.207.61:39749)
0808 07:06:43.007256 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:43.007258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:43.007270 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:06:43.007301 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:43.007302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:43.007360 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:43.007447 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:43.007705 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:43.008328 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:43.009325 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:43.009546 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:43.009622 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:43.009902 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:43.010087 (+ 185us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:43.010160 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:43.010234 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 07:06:43.010261 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":66,"threads_started":1}
I20260808 07:06:43.016083 5758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.012880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48088 (local address 127.13.207.60:39609)
0808 07:06:43.013054 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:43.013056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:43.013066 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:06:43.013112 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:43.013113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:43.013162 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:43.013260 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:43.013565 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:43.014236 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:43.015139 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:43.015457 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:43.015536 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:43.015713 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:43.015832 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:43.015902 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:43.015926 (+ 24us) server_negotiation.cc:299] Negotiation successful
0808 07:06:43.015950 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":87,"threads_started":1}
I20260808 07:06:43.021412 5752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.018235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44296 (local address 127.13.207.61:39749)
0808 07:06:43.018291 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:43.018294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:43.018428 (+ 134us) server_negotiation.cc:407] Connection header received
0808 07:06:43.018551 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:43.018552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:43.018568 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:43.018601 (+ 33us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:43.018883 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:43.019649 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:43.020488 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:43.020808 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:43.020890 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:43.021064 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:43.021106 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:43.021160 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:43.021264 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 07:06:43.021302 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:06:43.021605 5758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.018273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48090 (local address 127.13.207.60:39609)
0808 07:06:43.018432 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:43.018434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:43.018558 (+ 124us) server_negotiation.cc:407] Connection header received
0808 07:06:43.018588 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:43.018590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:43.018601 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:43.018657 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:43.018863 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:43.019627 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:43.020633 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:43.020972 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:43.021093 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:43.021272 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:43.021311 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:43.021363 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:43.021468 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 07:06:43.021495 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260808 07:06:43.022253 5763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.018112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36222 (local address 127.13.207.62:45073)
0808 07:06:43.018314 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:43.018316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:43.018498 (+ 182us) server_negotiation.cc:407] Connection header received
0808 07:06:43.018606 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:43.018607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:43.018679 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:43.018807 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:43.018933 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:43.019439 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:43.020615 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:43.020960 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:43.021052 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:43.021415 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:43.021872 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:43.021944 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:43.022135 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 07:06:43.022166 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":57,"threads_started":1}
I20260808 07:06:43.024240 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 5388
I20260808 07:06:43.031580 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 5465
I20260808 07:06:43.038753 14140 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzjmV4g/build/release/bin/kudu with pid 5543
2026-08-08T07:06:43Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3094 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6244 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (309990 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST